mirror of
https://github.com/Kaelio/ktx.git
synced 2026-06-25 08:48:08 +02:00
chore(workspace): gate dead-code with knip production mode (#196)
* refactor(workspace): relocate @ktx/llm source into packages/cli/src/llm * refactor(workspace): rewrite @ktx/llm imports to relative paths * refactor(workspace): fold internal packages into cli * chore(workspace): gate dead-code with knip production mode Turn on production-mode knip plus an autofix run in pre-commit and the `pnpm dead-code` script, document the `/** @internal */` convention for test-only exports in AGENTS.md, annotate test-only exports across the CLI with that JSDoc, and drop dead exports/wrappers the new gate surfaced (e.g. `cli-project.ts`, `lookerRuntimeSourceToFileAdapterSource`, `createLocalScanEnrichmentProvidersFromConfig`, `PGLITE_OWNER_PROCESS_BACKEND_CAPABILITIES`, stale type re-exports). Replace the loose `ignoreIssues` allowlist in `knip.json` with explicit production entries so cross-package barrel leaks are caught. * refactor(cli): delete internal barrel index.ts files The 34 `index.ts` re-export barrels inside `packages/cli/src/` were holdovers from the pre-fold multi-workspace structure. Post-fold-in they served no production purpose: external consumers go through the single package main entry, and in-repo callers mostly imported through them only because the path was short. Internally, knip flagged most barrel re-exports as production-dead (only reached via tests). This change: - Deletes every internal barrel except `packages/cli/src/index.ts` (the published package entry). - Rewrites ~270 source/test files to import each name directly from the file that defines it. - Moves `tools/warehouse-verification/index.ts` to `create-warehouse-verification-tools.ts` (the function it defined locally) and updates its single consumer. - Renames `search/backend-conformance.ts` → `.test-utils.ts` to match the existing test-helper file convention. - Deletes 13 dead test-only chains (dbt-descriptions/*, live-database/extracted-schema, live-database/structural-sync, relationship-* feedback/review chain) plus their tests and a cascading orphan integration test. - Updates test mocks that pointed at deleted barrel paths (notion-client, connector barrels in scan/local-scan-connectors tests) to mock the source files instead. - Points the maintainer benchmark script (`scripts/relationship-benchmark-report.mjs`) at source files instead of `dist/context/scan/index.js`. - Drops the barrel `!` entries from `knip.json`; adds explicit production entries only for the benchmark code reached via dist by the maintainer script. Net: 413 files changed, ~1.2k insertions, ~9.4k deletions. `pnpm run dead-code` (Biome + knip default + knip production) and `pnpm run type-check` are clean; 2277 tests pass. * refactor(workspace): rename @ktx/cli to @kaelio/ktx and pack it directly Promote the CLI workspace package to the public name `@kaelio/ktx` and drop the separate `scripts/build-public-npm-package.mjs` wrapper. The CLI package is now publishable in place (`publishConfig.access: public`, `provenance: true`), so artifact packing uses `pnpm pack` against `packages/cli/` instead of assembling a parallel package tree. Updates all workspace filter invocations, docs, tests, and release readiness checks to reference the new package name, and folds the tarball-name helper into `scripts/public-npm-release-metadata.mjs`. * docs: align "agent clients" and "data agents" terminology Replace "client agents" with "agent clients" and "database agents" with "data agents" across AGENTS.md, README.md, the docs-site copy, and the matching setup-agents test description, matching the canonical vocabulary in docs/terminology.md. Also moves packages/cli/tsconfig.json's tsBuildInfoFile from node_modules/.cache/ to dist/.tsbuildinfo so incremental builds survive node_modules reinstalls. * refactor(release): single source of truth for package version Make packages/cli/package.json the single source of truth for the @kaelio/ktx version. publicNpmPackageVersion() now reads it directly, so artifact filenames, release-readiness checks, and the Python wheel version all derive from one field. The duplicate release-policy.json.publicNpmPackageVersion is removed. Previously the two fields could drift: tarballs were named kaelio-ktx-0.4.1.tgz while internally containing @kaelio/ktx@0.0.0-private. - update-public-release-version.mjs rewrites both Python pyproject.toml files (ktx-daemon, ktx-sl) alongside the npm package.jsons, normalizing the version for PEP 440 (e.g. 0.1.0-rc.2 -> 0.1.0rc2). - semantic-release-config.cjs adds the two pyproject.toml files to @semantic-release/git assets so the release commit back to main carries every version source in lockstep. - The six "?? '0.0.0-private'" fallback literals across the CLI are replaced with "?? getKtxCliPackageInfo().version", and createDefaultKtxMcpServer makes its version arg required. - docs/release.md describes the actual commit-back model: the dev tree always reflects the most recent release; no sentinel pin to maintain. Verified: pnpm run artifacts:build now produces kaelio-ktx-0.4.1.tgz and kaelio_ktx-0.4.1-py3-none-any.whl with @kaelio/ktx@0.4.1 inside. Full type-check, dead-code, and 2287 vitests + 173 script tests pass. * refactor(cli): inject embedding provider resolution and detect sentence-transformers runtime Make resolveProjectEmbeddingProvider and runtimeIo injectable in ingest and scan command entrypoints so tests can stub them, and teach resolvePublicIngestRuntimeRequirements to flag the local-embeddings runtime feature when ktx.yaml selects sentence-transformers. * chore(cli): mark buildLocalStatsStatus and LocalStatsStatus as @internal Both symbols are consumed only by status-project.test.ts. Annotating with /** @internal */ keeps knip's production-mode check clean without changing runtime behavior. * fix(cli): use real package metadata in print-command-tree The stubbed package name embedded a forbidden product identifier that tripped the boundary check in CI. Read the metadata from package.json instead — keeps the rendered tree unchanged and removes a duplicate source of truth. * feat(cli): show embedding coverage in `ktx status`, drop duplicate disk counts Inline `(N embedded)` next to the Wiki scope counts and Semantic-layer source counts, computed with `SUM(embedding_json IS NOT NULL)` over `knowledge_pages` and `local_sl_sources`. Rename the "Knowledge" label to "Wiki" (canonical per `docs/terminology.md`) and rename the matching `localStats.knowledgePages` field to `localStats.wikiPages`. Drop `wiki=N md` and `semantic-layer=N yaml` from the Disk row — those duplicated the per-surface rows above. Disk now reports only actual byte usage (db, cache, raw-sources). The unused `wikiGlobalMarkdownCount` / `semanticLayerYamlCount` fields, the `isMarkdownEntry` / `isYamlEntry` helpers, and the `filter` arg on `summarizeDir` are removed.
This commit is contained in:
parent
a1cfb03d73
commit
2366b00301
1002 changed files with 2286 additions and 12051 deletions
|
|
@ -0,0 +1,44 @@
|
|||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { ToolContext } from '../../../context/tools/base-tool.js';
|
||||
import { WikiListTagsTool } from './wiki-list-tags.tool.js';
|
||||
|
||||
describe('WikiListTagsTool', () => {
|
||||
const baseContext: ToolContext = { sourceId: 's', messageId: 'm', userId: 'u' };
|
||||
|
||||
it("returns distinct sorted tags across the user's visible pages", async () => {
|
||||
const pagesRepository = {
|
||||
listPagesForUser: vi.fn().mockResolvedValue([
|
||||
{ scope: 'GLOBAL', scope_id: null, page_key: 'k1', tags: ['metrics', 'finance'] },
|
||||
{ scope: 'USER', scope_id: 'u', page_key: 'k2', tags: ['metrics'] },
|
||||
]),
|
||||
};
|
||||
const tool = new WikiListTagsTool(pagesRepository as any);
|
||||
|
||||
const result = await tool.call({}, baseContext);
|
||||
expect(result.markdown).toContain('finance');
|
||||
expect(result.markdown).toContain('metrics');
|
||||
expect(result.structured.tags).toEqual(['finance', 'metrics']);
|
||||
});
|
||||
|
||||
it('lists tags from historic-SQL indexed pages with flat wiki keys', async () => {
|
||||
const pagesRepository = {
|
||||
listPagesForUser: vi.fn().mockResolvedValue([
|
||||
{ scope: 'GLOBAL', scope_id: null, page_key: 'company-overview', tags: ['notion'] },
|
||||
{ scope: 'GLOBAL', scope_id: null, page_key: 'historic-sql-revenue-pattern', tags: ['historic-sql', 'pattern'] },
|
||||
]),
|
||||
};
|
||||
const tool = new WikiListTagsTool(pagesRepository as any);
|
||||
|
||||
const result = await tool.call({}, baseContext);
|
||||
|
||||
expect(result.structured.tags).toEqual(['historic-sql', 'notion', 'pattern']);
|
||||
});
|
||||
|
||||
it('returns a friendly message when no pages have tags', async () => {
|
||||
const pagesRepository = { listPagesForUser: vi.fn().mockResolvedValue([]) };
|
||||
const tool = new WikiListTagsTool(pagesRepository as any);
|
||||
|
||||
const result = await tool.call({}, baseContext);
|
||||
expect(result.markdown).toMatch(/no tags/i);
|
||||
});
|
||||
});
|
||||
41
packages/cli/src/context/wiki/tools/wiki-list-tags.tool.ts
Normal file
41
packages/cli/src/context/wiki/tools/wiki-list-tags.tool.ts
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
import { z } from 'zod';
|
||||
import type { KnowledgeIndexPort } from '../ports.js';
|
||||
import { BaseTool, type ToolContext, type ToolOutput } from '../../../context/tools/base-tool.js';
|
||||
|
||||
const wikiListTagsInputSchema = z.object({});
|
||||
|
||||
type WikiListTagsInput = z.infer<typeof wikiListTagsInputSchema>;
|
||||
|
||||
export class WikiListTagsTool extends BaseTool<typeof wikiListTagsInputSchema> {
|
||||
readonly name = 'wiki_list_tags';
|
||||
|
||||
constructor(private readonly pagesRepository: KnowledgeIndexPort) {
|
||||
super();
|
||||
}
|
||||
|
||||
get description(): string {
|
||||
return `<purpose>
|
||||
List distinct topic tags across all wiki pages visible to the user.
|
||||
Call before writing a new page so you can reuse existing tags consistently instead of coining near-duplicates.
|
||||
</purpose>`;
|
||||
}
|
||||
|
||||
get inputSchema() {
|
||||
return wikiListTagsInputSchema;
|
||||
}
|
||||
|
||||
async call(_input: WikiListTagsInput, context: ToolContext): Promise<ToolOutput<{ tags: string[] }>> {
|
||||
const pages = await this.pagesRepository.listPagesForUser(context.userId);
|
||||
const set = new Set<string>();
|
||||
for (const p of pages) {
|
||||
for (const t of p.tags) {
|
||||
set.add(t);
|
||||
}
|
||||
}
|
||||
const tags = [...set].sort();
|
||||
return {
|
||||
markdown: tags.length === 0 ? '(no tags in use yet)' : tags.join(', '),
|
||||
structured: { tags },
|
||||
};
|
||||
}
|
||||
}
|
||||
80
packages/cli/src/context/wiki/tools/wiki-read.tool.test.ts
Normal file
80
packages/cli/src/context/wiki/tools/wiki-read.tool.test.ts
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { ToolSession } from '../../../context/tools/tool-session.js';
|
||||
import { createTouchedSlSources } from '../../../context/tools/touched-sl-sources.js';
|
||||
import type { ToolContext } from '../../../context/tools/base-tool.js';
|
||||
import { WikiReadTool } from './wiki-read.tool.js';
|
||||
|
||||
describe('WikiReadTool', () => {
|
||||
const baseContext: ToolContext = { sourceId: 's', messageId: 'm', userId: 'u' };
|
||||
|
||||
it('reads from the session wiki service when a worktree-scoped ingest session is present', async () => {
|
||||
const rootWikiService = { readPageForUser: vi.fn().mockResolvedValue(null) };
|
||||
const sessionWikiService = {
|
||||
readPageForUser: vi.fn().mockResolvedValue({
|
||||
pageKey: 'staged-page',
|
||||
scope: 'GLOBAL',
|
||||
frontmatter: { summary: 'Staged', tags: ['notion'], refs: ['related'] },
|
||||
content: 'A page written earlier in the same ingest worktree.',
|
||||
}),
|
||||
};
|
||||
const pagesRepository = { findPageByKey: vi.fn().mockResolvedValue({ id: 'page-1' }), incrementUsageCount: vi.fn() };
|
||||
const tool = new WikiReadTool(rootWikiService as any, pagesRepository as any);
|
||||
const session: ToolSession = {
|
||||
connectionId: 'c',
|
||||
isWorktreeScoped: true,
|
||||
preHead: null,
|
||||
touchedSlSources: createTouchedSlSources(),
|
||||
actions: [],
|
||||
semanticLayerService: {} as any,
|
||||
wikiService: sessionWikiService as any,
|
||||
configService: {} as any,
|
||||
gitService: {} as any,
|
||||
};
|
||||
|
||||
const result = await tool.call({ key: 'staged-page' }, { ...baseContext, session });
|
||||
|
||||
expect(rootWikiService.readPageForUser).not.toHaveBeenCalled();
|
||||
expect(sessionWikiService.readPageForUser).toHaveBeenCalledWith('u', 'staged-page');
|
||||
expect(result.structured).toMatchObject({ found: true, blockKey: 'staged-page', scope: 'GLOBAL' });
|
||||
expect(result.markdown).toContain('A page written earlier in the same ingest worktree.');
|
||||
});
|
||||
|
||||
it('rejects slash-delimited page keys with a flat-key suggestion', async () => {
|
||||
const rootWikiService = { readPageForUser: vi.fn().mockResolvedValue(null) };
|
||||
const pagesRepository = { findPageByKey: vi.fn(), incrementUsageCount: vi.fn() };
|
||||
const tool = new WikiReadTool(rootWikiService as any, pagesRepository as any);
|
||||
|
||||
const result = await tool.call({ key: 'orbit/company-overview' }, baseContext);
|
||||
|
||||
expect(result.structured).toEqual({
|
||||
blockKey: 'orbit/company-overview',
|
||||
content: '',
|
||||
scope: '',
|
||||
found: false,
|
||||
});
|
||||
expect(result.markdown).toContain(
|
||||
'Invalid wiki key "orbit/company-overview". Wiki keys must be flat; use "orbit-company-overview".',
|
||||
);
|
||||
expect(rootWikiService.readPageForUser).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not append derived refs to the editable markdown body', async () => {
|
||||
const rootWikiService = {
|
||||
readPageForUser: vi.fn().mockResolvedValue({
|
||||
pageKey: 'orbit-how-we-work',
|
||||
scope: 'GLOBAL',
|
||||
frontmatter: { summary: 'How we work', tags: ['policy'], refs: ['orbit-company-overview'] },
|
||||
content: '## How We Work\n\nUse written-first operating norms.',
|
||||
}),
|
||||
};
|
||||
const pagesRepository = { findPageByKey: vi.fn().mockResolvedValue(null), incrementUsageCount: vi.fn() };
|
||||
const tool = new WikiReadTool(rootWikiService as any, pagesRepository as any);
|
||||
|
||||
const result = await tool.call({ key: 'orbit-how-we-work' }, baseContext);
|
||||
|
||||
expect(result.markdown).toBe('## How We Work\n\nUse written-first operating norms.');
|
||||
expect(result.markdown).not.toContain('See also');
|
||||
expect(result.markdown).not.toContain('[[orbit-company-overview]]');
|
||||
expect(result.structured.refs).toEqual(['orbit-company-overview']);
|
||||
});
|
||||
});
|
||||
86
packages/cli/src/context/wiki/tools/wiki-read.tool.ts
Normal file
86
packages/cli/src/context/wiki/tools/wiki-read.tool.ts
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
import { z } from 'zod';
|
||||
import type { KnowledgeIndexPort } from '../ports.js';
|
||||
import { KnowledgeWikiService } from '../../../context/wiki/knowledge-wiki.service.js';
|
||||
import { validateFlatWikiKey } from '../keys.js';
|
||||
import { BaseTool, type ToolContext, type ToolOutput } from '../../../context/tools/base-tool.js';
|
||||
|
||||
const WikiReadInputSchema = z.object({
|
||||
key: z
|
||||
.string()
|
||||
.describe('The block_key to read. Check the <knowledge_index> in the system prompt for available keys.'),
|
||||
});
|
||||
|
||||
type WikiReadInput = z.infer<typeof WikiReadInputSchema>;
|
||||
|
||||
interface WikiReadStructured {
|
||||
blockKey: string;
|
||||
content: string;
|
||||
scope: string;
|
||||
found: boolean;
|
||||
tags?: string[];
|
||||
refs?: string[];
|
||||
}
|
||||
|
||||
export class WikiReadTool extends BaseTool<typeof WikiReadInputSchema> {
|
||||
readonly name = 'wiki_read';
|
||||
|
||||
constructor(
|
||||
private readonly wikiService: KnowledgeWikiService,
|
||||
private readonly pagesRepository: KnowledgeIndexPort,
|
||||
) {
|
||||
super();
|
||||
}
|
||||
|
||||
get description(): string {
|
||||
return (
|
||||
'Load the full content of a knowledge block by its key. ' +
|
||||
'Use this to retrieve detailed rules, preferences, or definitions listed in the <knowledge_index>. ' +
|
||||
'The markdown output is the exact stored page body; use it verbatim for wiki_write replacements. ' +
|
||||
'Call this when the user query relates to a topic covered by an available knowledge block.'
|
||||
);
|
||||
}
|
||||
|
||||
get inputSchema() {
|
||||
return WikiReadInputSchema;
|
||||
}
|
||||
|
||||
async call(input: WikiReadInput, context: ToolContext): Promise<ToolOutput<WikiReadStructured>> {
|
||||
const keyValidation = validateFlatWikiKey(input.key);
|
||||
if (!keyValidation.ok) {
|
||||
return {
|
||||
markdown: keyValidation.error,
|
||||
structured: { blockKey: input.key, content: '', scope: '', found: false },
|
||||
};
|
||||
}
|
||||
const wikiService = context.session?.wikiService ?? this.wikiService;
|
||||
const page = await wikiService.readPageForUser(context.userId, input.key);
|
||||
|
||||
if (!page) {
|
||||
return {
|
||||
markdown: `No knowledge block found with key "${input.key}".`,
|
||||
structured: { blockKey: input.key, content: '', scope: '', found: false },
|
||||
};
|
||||
}
|
||||
|
||||
const indexEntry = await this.pagesRepository.findPageByKey(
|
||||
page.scope,
|
||||
page.scope === 'USER' ? context.userId : null,
|
||||
input.key,
|
||||
);
|
||||
if (indexEntry?.id) {
|
||||
void this.pagesRepository.incrementUsageCount([indexEntry.id]);
|
||||
}
|
||||
|
||||
return {
|
||||
markdown: page.content,
|
||||
structured: {
|
||||
blockKey: page.pageKey,
|
||||
content: page.content,
|
||||
scope: page.scope,
|
||||
found: true,
|
||||
tags: page.frontmatter.tags,
|
||||
refs: page.frontmatter.refs,
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
109
packages/cli/src/context/wiki/tools/wiki-remove.tool.test.ts
Normal file
109
packages/cli/src/context/wiki/tools/wiki-remove.tool.test.ts
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { ToolSession } from '../../../context/tools/tool-session.js';
|
||||
import { createTouchedSlSources } from '../../../context/tools/touched-sl-sources.js';
|
||||
import type { ToolContext } from '../../../context/tools/base-tool.js';
|
||||
import { WikiRemoveTool } from './wiki-remove.tool.js';
|
||||
|
||||
describe('WikiRemoveTool', () => {
|
||||
const baseContext: ToolContext = { sourceId: 's', messageId: 'm', userId: 'u' };
|
||||
|
||||
it('removes an existing page when no session is present', async () => {
|
||||
const wikiService = {
|
||||
deletePage: vi.fn().mockResolvedValue(undefined),
|
||||
deleteFromIndex: vi.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
const pagesRepository = {
|
||||
findPageByKey: vi.fn().mockResolvedValue({ page_key: 'old' }),
|
||||
};
|
||||
const knowledgeRepository = { createEvent: vi.fn().mockResolvedValue(undefined) };
|
||||
const tool = new WikiRemoveTool(wikiService as any, pagesRepository as any, knowledgeRepository as any);
|
||||
const result = await tool.call({ key: 'old' } as any, baseContext);
|
||||
expect(wikiService.deletePage).toHaveBeenCalledTimes(1);
|
||||
expect(wikiService.deleteFromIndex).toHaveBeenCalledTimes(1);
|
||||
expect(result.markdown).toMatch(/removed/i);
|
||||
});
|
||||
|
||||
it('rejects slash-delimited page keys with a flat-key suggestion', async () => {
|
||||
const wikiService = {
|
||||
deletePage: vi.fn().mockResolvedValue(undefined),
|
||||
deleteFromIndex: vi.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
const pagesRepository = { findPageByKey: vi.fn().mockResolvedValue({ page_key: 'old' }) };
|
||||
const knowledgeRepository = { createEvent: vi.fn().mockResolvedValue(undefined) };
|
||||
const tool = new WikiRemoveTool(wikiService as any, pagesRepository as any, knowledgeRepository as any);
|
||||
|
||||
const result = await tool.call({ key: 'orbit/company-overview' } as any, baseContext);
|
||||
|
||||
expect(result.structured).toEqual({ success: false, key: 'orbit/company-overview' });
|
||||
expect(result.markdown).toContain(
|
||||
'Invalid wiki key "orbit/company-overview". Wiki keys must be flat; use "orbit-company-overview".',
|
||||
);
|
||||
expect(pagesRepository.findPageByKey).not.toHaveBeenCalled();
|
||||
expect(wikiService.deletePage).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('skips deleteFromIndex when session is worktree-scoped', async () => {
|
||||
const wikiService = {
|
||||
readPage: vi.fn().mockResolvedValue({ pageKey: 'old', frontmatter: { summary: 'Old' }, content: 'body' }),
|
||||
deletePage: vi.fn().mockResolvedValue(undefined),
|
||||
deleteFromIndex: vi.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
const pagesRepository = { findPageByKey: vi.fn().mockResolvedValue({ page_key: 'old' }) };
|
||||
const knowledgeRepository = { createEvent: vi.fn().mockResolvedValue(undefined) };
|
||||
const tool = new WikiRemoveTool(wikiService as any, pagesRepository as any, knowledgeRepository as any);
|
||||
const session: ToolSession = {
|
||||
connectionId: 'c',
|
||||
isWorktreeScoped: true,
|
||||
preHead: null,
|
||||
touchedSlSources: createTouchedSlSources(),
|
||||
actions: [],
|
||||
semanticLayerService: {} as any,
|
||||
wikiService: wikiService as any,
|
||||
configService: {} as any,
|
||||
gitService: {} as any,
|
||||
};
|
||||
await tool.call({ key: 'old' } as any, { ...baseContext, session });
|
||||
expect(wikiService.deletePage).toHaveBeenCalledTimes(1);
|
||||
expect(wikiService.deleteFromIndex).not.toHaveBeenCalled();
|
||||
expect(session.actions).toContainEqual(expect.objectContaining({ target: 'wiki', type: 'removed', key: 'old' }));
|
||||
});
|
||||
|
||||
it('finds pages through the session wiki service even when the shared index has not seen the worktree write', async () => {
|
||||
const wikiService = {
|
||||
readPage: vi.fn().mockResolvedValue({ pageKey: 'staged', frontmatter: { summary: 'Staged' }, content: 'body' }),
|
||||
deletePage: vi.fn().mockResolvedValue(undefined),
|
||||
deleteFromIndex: vi.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
const pagesRepository = { findPageByKey: vi.fn().mockResolvedValue(null) };
|
||||
const knowledgeRepository = { createEvent: vi.fn().mockResolvedValue(undefined) };
|
||||
const tool = new WikiRemoveTool(wikiService as any, pagesRepository as any, knowledgeRepository as any);
|
||||
const session: ToolSession = {
|
||||
connectionId: 'c',
|
||||
isWorktreeScoped: true,
|
||||
preHead: null,
|
||||
touchedSlSources: createTouchedSlSources(),
|
||||
actions: [],
|
||||
semanticLayerService: {} as any,
|
||||
wikiService: wikiService as any,
|
||||
configService: {} as any,
|
||||
gitService: {} as any,
|
||||
};
|
||||
|
||||
const result = await tool.call({ key: 'staged' } as any, { ...baseContext, session });
|
||||
|
||||
expect(pagesRepository.findPageByKey).not.toHaveBeenCalled();
|
||||
expect(wikiService.readPage).toHaveBeenCalledWith('GLOBAL', null, 'staged');
|
||||
expect(wikiService.deletePage).toHaveBeenCalledTimes(1);
|
||||
expect(result.structured).toEqual({ success: true, key: 'staged' });
|
||||
});
|
||||
|
||||
it('returns a friendly message when the page does not exist', async () => {
|
||||
const wikiService = { deletePage: vi.fn(), deleteFromIndex: vi.fn() };
|
||||
const pagesRepository = { findPageByKey: vi.fn().mockResolvedValue(null) };
|
||||
const knowledgeRepository = { createEvent: vi.fn() };
|
||||
const tool = new WikiRemoveTool(wikiService as any, pagesRepository as any, knowledgeRepository as any);
|
||||
const result = await tool.call({ key: 'missing' } as any, baseContext);
|
||||
expect(result.structured.success).toBe(false);
|
||||
expect(result.markdown).toMatch(/not found/i);
|
||||
});
|
||||
});
|
||||
108
packages/cli/src/context/wiki/tools/wiki-remove.tool.ts
Normal file
108
packages/cli/src/context/wiki/tools/wiki-remove.tool.ts
Normal file
|
|
@ -0,0 +1,108 @@
|
|||
import { z } from 'zod';
|
||||
import type { KnowledgeIndexPort } from '../ports.js';
|
||||
import type { KnowledgeEventPort } from '../ports.js';
|
||||
type BlockScope = 'GLOBAL' | 'USER';
|
||||
import { KnowledgeWikiService } from '../../../context/wiki/knowledge-wiki.service.js';
|
||||
import { validateFlatWikiKey } from '../keys.js';
|
||||
import { BaseTool, type ToolContext, type ToolOutput } from '../../../context/tools/base-tool.js';
|
||||
import { validateActionRawPaths } from '../../../context/tools/action-raw-paths.js';
|
||||
|
||||
const SYSTEM_AUTHOR = 'System User';
|
||||
const SYSTEM_EMAIL = 'system@example.com';
|
||||
|
||||
const wikiRemoveInputSchema = z.object({
|
||||
key: z.string().describe('The page key to remove'),
|
||||
rawPaths: z
|
||||
.array(z.string().min(1))
|
||||
.optional()
|
||||
.describe('In ingest sessions, raw source file paths that directly support this removal.'),
|
||||
});
|
||||
|
||||
type WikiRemoveInput = z.infer<typeof wikiRemoveInputSchema>;
|
||||
|
||||
interface WikiRemoveStructured {
|
||||
success: boolean;
|
||||
key: string;
|
||||
}
|
||||
|
||||
export class WikiRemoveTool extends BaseTool<typeof wikiRemoveInputSchema> {
|
||||
readonly name = 'wiki_remove';
|
||||
|
||||
constructor(
|
||||
private readonly wikiService: KnowledgeWikiService,
|
||||
private readonly pagesRepository: KnowledgeIndexPort,
|
||||
private readonly knowledgeRepository: KnowledgeEventPort,
|
||||
) {
|
||||
super();
|
||||
}
|
||||
|
||||
get description(): string {
|
||||
return `<purpose>Remove a wiki page that is no longer relevant.</purpose>`;
|
||||
}
|
||||
|
||||
get inputSchema() {
|
||||
return wikiRemoveInputSchema;
|
||||
}
|
||||
|
||||
async call(input: WikiRemoveInput, context: ToolContext): Promise<ToolOutput<WikiRemoveStructured>> {
|
||||
const wikiService = context.session?.wikiService ?? this.wikiService;
|
||||
const writesGlobal = !!context.session;
|
||||
const skipIndex = context.session?.isWorktreeScoped === true;
|
||||
const keyValidation = validateFlatWikiKey(input.key);
|
||||
if (!keyValidation.ok) {
|
||||
return {
|
||||
markdown: keyValidation.error,
|
||||
structured: { success: false, key: input.key },
|
||||
};
|
||||
}
|
||||
const rawPathValidation = validateActionRawPaths(context.session, input.rawPaths);
|
||||
if (!rawPathValidation.ok) {
|
||||
return {
|
||||
markdown: `Error: ${rawPathValidation.error}`,
|
||||
structured: { success: false, key: input.key },
|
||||
};
|
||||
}
|
||||
|
||||
const scope: BlockScope = writesGlobal ? 'GLOBAL' : 'USER';
|
||||
const scopeId = scope === 'USER' ? context.userId : null;
|
||||
|
||||
const existing = context.session
|
||||
? await wikiService.readPage(scope, scopeId, input.key)
|
||||
: await this.pagesRepository.findPageByKey(scope, scopeId, input.key);
|
||||
if (!existing) {
|
||||
return {
|
||||
markdown: `Page "${input.key}" not found.`,
|
||||
structured: { success: false, key: input.key },
|
||||
};
|
||||
}
|
||||
|
||||
await wikiService.deletePage(scope, scopeId, input.key, SYSTEM_AUTHOR, SYSTEM_EMAIL);
|
||||
if (!skipIndex) {
|
||||
await wikiService.deleteFromIndex(scope, scopeId, input.key);
|
||||
}
|
||||
|
||||
await this.knowledgeRepository.createEvent({
|
||||
blockId: null,
|
||||
eventType: 'BLOCK_REMOVED',
|
||||
actorId: context.userId,
|
||||
chatId: null,
|
||||
messageId: null,
|
||||
payload: { removedKey: input.key, blockKey: input.key },
|
||||
});
|
||||
|
||||
if (context.session) {
|
||||
context.session.actions.push({
|
||||
target: 'wiki',
|
||||
type: 'removed',
|
||||
key: input.key,
|
||||
detail: `Removed page "${input.key}"`,
|
||||
...(rawPathValidation.rawPaths ? { rawPaths: rawPathValidation.rawPaths } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
return {
|
||||
markdown: `Page "${input.key}" removed.`,
|
||||
structured: { success: true, key: input.key },
|
||||
};
|
||||
}
|
||||
}
|
||||
41
packages/cli/src/context/wiki/tools/wiki-search.tool.test.ts
Normal file
41
packages/cli/src/context/wiki/tools/wiki-search.tool.test.ts
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { WikiSearchTool } from './wiki-search.tool.js';
|
||||
|
||||
describe('WikiSearchTool', () => {
|
||||
it('searches through the injected wiki adapter port', async () => {
|
||||
const search = vi.fn(async () => ({
|
||||
results: [
|
||||
{
|
||||
key: 'metrics-revenue',
|
||||
path: 'wiki/global/metrics-revenue.md',
|
||||
scope: 'GLOBAL' as const,
|
||||
summary: 'Revenue metric definition',
|
||||
score: 0.02459016393442623,
|
||||
matchReasons: ['lexical' as const, 'token' as const],
|
||||
},
|
||||
],
|
||||
totalFound: 1,
|
||||
}));
|
||||
const tool = new WikiSearchTool({ search });
|
||||
|
||||
const result = await tool.call(
|
||||
{ query: 'paid order', limit: 5 },
|
||||
{ sourceId: 'test', messageId: 'message-1', userId: 'agent' },
|
||||
);
|
||||
|
||||
expect(search).toHaveBeenCalledWith({ userId: 'agent', query: 'paid order', limit: 5 });
|
||||
expect(result.structured).toEqual({
|
||||
results: [
|
||||
{
|
||||
blockKey: 'metrics-revenue',
|
||||
path: 'wiki/global/metrics-revenue.md',
|
||||
summary: 'Revenue metric definition',
|
||||
score: 0.02459016393442623,
|
||||
matchReasons: ['lexical', 'token'],
|
||||
},
|
||||
],
|
||||
totalFound: 1,
|
||||
});
|
||||
expect(result.markdown).toContain('**metrics-revenue**');
|
||||
});
|
||||
});
|
||||
96
packages/cli/src/context/wiki/tools/wiki-search.tool.ts
Normal file
96
packages/cli/src/context/wiki/tools/wiki-search.tool.ts
Normal file
|
|
@ -0,0 +1,96 @@
|
|||
import { z } from 'zod';
|
||||
import { BaseTool, type ToolContext, type ToolOutput } from '../../../context/tools/base-tool.js';
|
||||
import type { WikiSearchLaneSummary, WikiSearchMatchReason } from '../types.js';
|
||||
|
||||
const WikiSearchInputSchema = z.object({
|
||||
query: z.string().describe('Natural language search query to find relevant knowledge blocks.'),
|
||||
limit: z.number().optional().default(10).describe('Maximum number of results to return (default 10).'),
|
||||
});
|
||||
|
||||
type WikiSearchInput = z.infer<typeof WikiSearchInputSchema>;
|
||||
|
||||
interface WikiSearchResult {
|
||||
blockKey: string;
|
||||
path: string;
|
||||
summary: string;
|
||||
score: number;
|
||||
matchReasons?: WikiSearchMatchReason[];
|
||||
lanes?: WikiSearchLaneSummary[];
|
||||
}
|
||||
|
||||
interface WikiSearchStructured {
|
||||
results: WikiSearchResult[];
|
||||
totalFound: number;
|
||||
}
|
||||
|
||||
/** @internal */
|
||||
export interface WikiSearchAdapterPort {
|
||||
search(input: { userId: string; query: string; limit: number }): Promise<{
|
||||
results: Array<{
|
||||
key: string;
|
||||
path: string;
|
||||
summary: string;
|
||||
score: number;
|
||||
matchReasons?: WikiSearchMatchReason[];
|
||||
lanes?: WikiSearchLaneSummary[];
|
||||
}>;
|
||||
totalFound: number;
|
||||
}>;
|
||||
}
|
||||
|
||||
export class WikiSearchTool extends BaseTool<typeof WikiSearchInputSchema> {
|
||||
readonly name = 'wiki_search';
|
||||
|
||||
constructor(private readonly searchAdapter: WikiSearchAdapterPort) {
|
||||
super();
|
||||
}
|
||||
|
||||
get description(): string {
|
||||
return (
|
||||
'Search knowledge blocks. Active lanes vary by project storage: ' +
|
||||
'projects on sqlite-fts5 storage use hybrid lexical + token + semantic matching, ' +
|
||||
'others fall back to token-only matching. ' +
|
||||
'Inspect `lanes` and `matchReasons` on each result to see which lanes contributed. ' +
|
||||
'Use this when you need to find knowledge on a topic not visible in the discovery index. ' +
|
||||
'Returns ranked summaries — use wiki_read to load the full content of specific results.'
|
||||
);
|
||||
}
|
||||
|
||||
get inputSchema() {
|
||||
return WikiSearchInputSchema;
|
||||
}
|
||||
|
||||
async call(input: WikiSearchInput, context: ToolContext): Promise<ToolOutput<WikiSearchStructured>> {
|
||||
const response = await this.searchAdapter.search({
|
||||
userId: context.userId,
|
||||
query: input.query,
|
||||
limit: input.limit,
|
||||
});
|
||||
|
||||
if (response.results.length === 0) {
|
||||
return {
|
||||
markdown: `No knowledge blocks found matching "${input.query}".`,
|
||||
structured: { results: [], totalFound: 0 },
|
||||
};
|
||||
}
|
||||
|
||||
const lines = response.results.map((r, i) => `${i + 1}. **${r.key}**: ${r.summary}`);
|
||||
|
||||
const structured: WikiSearchStructured = {
|
||||
results: response.results.map((r) => ({
|
||||
blockKey: r.key,
|
||||
path: r.path,
|
||||
summary: r.summary,
|
||||
score: r.score,
|
||||
matchReasons: r.matchReasons,
|
||||
lanes: r.lanes,
|
||||
})),
|
||||
totalFound: response.totalFound,
|
||||
};
|
||||
|
||||
return {
|
||||
markdown: `Found ${response.results.length} knowledge block(s):\n\n${lines.join('\n')}`,
|
||||
structured,
|
||||
};
|
||||
}
|
||||
}
|
||||
344
packages/cli/src/context/wiki/tools/wiki-write.tool.test.ts
Normal file
344
packages/cli/src/context/wiki/tools/wiki-write.tool.test.ts
Normal file
|
|
@ -0,0 +1,344 @@
|
|||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { ToolSession } from '../../../context/tools/tool-session.js';
|
||||
import { createTouchedSlSources } from '../../../context/tools/touched-sl-sources.js';
|
||||
import type { ToolContext } from '../../../context/tools/base-tool.js';
|
||||
import { WikiWriteTool } from './wiki-write.tool.js';
|
||||
|
||||
function makeTool(overrides: any = {}) {
|
||||
const wikiService = {
|
||||
readPage: vi.fn().mockResolvedValue(null),
|
||||
listPageKeys: vi.fn().mockResolvedValue([]),
|
||||
writePage: vi.fn().mockResolvedValue(undefined),
|
||||
syncSinglePage: vi.fn().mockResolvedValue(undefined),
|
||||
...overrides.wikiService,
|
||||
};
|
||||
const pagesRepository = {
|
||||
findPageByKey: vi.fn().mockResolvedValue(null),
|
||||
getUserPageCount: vi.fn().mockResolvedValue(0),
|
||||
...overrides.pagesRepository,
|
||||
};
|
||||
const knowledgeRepository = {
|
||||
createEvent: vi.fn().mockResolvedValue(undefined),
|
||||
...overrides.knowledgeRepository,
|
||||
};
|
||||
const tool = new WikiWriteTool(wikiService as any, pagesRepository as any, knowledgeRepository as any);
|
||||
return { tool, wikiService, pagesRepository, knowledgeRepository };
|
||||
}
|
||||
|
||||
describe('WikiWriteTool', () => {
|
||||
const baseContext: ToolContext = { sourceId: 's', messageId: 'm', userId: 'u' };
|
||||
|
||||
it('creates a new page and indexes it when no session is present', async () => {
|
||||
const { tool, wikiService } = makeTool();
|
||||
const result = await tool.call(
|
||||
{ key: 'leads-source', summary: 'Lead source definitions', content: '# Leads' } as any,
|
||||
baseContext,
|
||||
);
|
||||
expect(wikiService.writePage).toHaveBeenCalledTimes(1);
|
||||
expect(wikiService.syncSinglePage).toHaveBeenCalledTimes(1);
|
||||
expect(result.markdown).toMatch(/created/i);
|
||||
});
|
||||
|
||||
it('rejects slash-delimited page keys with a flat-key suggestion', async () => {
|
||||
const { tool, wikiService } = makeTool();
|
||||
const result = await tool.call(
|
||||
{ key: 'orbit/company-overview', summary: 'Company overview', content: '# Orbit' } as any,
|
||||
baseContext,
|
||||
);
|
||||
|
||||
expect(result.structured).toEqual({ success: false, key: 'orbit/company-overview' });
|
||||
expect(result.markdown).toContain(
|
||||
'Invalid wiki key "orbit/company-overview". Wiki keys must be flat; use "orbit-company-overview".',
|
||||
);
|
||||
expect(wikiService.readPage).not.toHaveBeenCalled();
|
||||
expect(wikiService.writePage).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('normalizes accidentally escaped markdown newlines before writing', async () => {
|
||||
const { tool, wikiService } = makeTool();
|
||||
|
||||
await tool.call(
|
||||
{
|
||||
key: 'large-contract-requesters',
|
||||
summary: 'Cross-schema Metabase query',
|
||||
content:
|
||||
'# Large Contract Requesters\\n\\n**Source card:** Metabase #110\\n\\n## SQL\\n\\n```sql\\nselect * from orbit_analytics.mart_account_segments\\n```\\n',
|
||||
} as any,
|
||||
baseContext,
|
||||
);
|
||||
|
||||
expect(wikiService.writePage.mock.calls[0][4]).toBe(
|
||||
'# Large Contract Requesters\n\n**Source card:** Metabase #110\n\n## SQL\n\n```sql\nselect * from orbit_analytics.mart_account_segments\n```\n',
|
||||
);
|
||||
expect(wikiService.syncSinglePage.mock.calls[0][4]).toBe(
|
||||
'# Large Contract Requesters\n\n**Source card:** Metabase #110\n\n## SQL\n\n```sql\nselect * from orbit_analytics.mart_account_segments\n```\n',
|
||||
);
|
||||
});
|
||||
|
||||
it('preserves intentional escaped newline examples in inline code', async () => {
|
||||
const { tool, wikiService } = makeTool();
|
||||
|
||||
await tool.call(
|
||||
{
|
||||
key: 'newline-token',
|
||||
summary: 'Escaped newline token',
|
||||
content: 'Use `\\n\\n` when documenting the literal separator.',
|
||||
} as any,
|
||||
baseContext,
|
||||
);
|
||||
|
||||
expect(wikiService.writePage.mock.calls[0][4]).toBe('Use `\\n\\n` when documenting the literal separator.');
|
||||
});
|
||||
|
||||
it('skips syncSinglePage when session is worktree-scoped', async () => {
|
||||
const { tool, wikiService } = makeTool();
|
||||
const session: ToolSession = {
|
||||
connectionId: 'conn-1',
|
||||
isWorktreeScoped: true,
|
||||
preHead: null,
|
||||
touchedSlSources: createTouchedSlSources(),
|
||||
actions: [],
|
||||
semanticLayerService: {} as any,
|
||||
wikiService: wikiService as any,
|
||||
configService: {} as any,
|
||||
gitService: {} as any,
|
||||
};
|
||||
const context: ToolContext = { ...baseContext, session };
|
||||
await tool.call({ key: 'k', summary: 's', content: '# x' } as any, context);
|
||||
expect(wikiService.writePage).toHaveBeenCalledTimes(1);
|
||||
expect(wikiService.syncSinglePage).not.toHaveBeenCalled();
|
||||
expect(session.actions).toContainEqual(expect.objectContaining({ target: 'wiki', type: 'created', key: 'k' }));
|
||||
});
|
||||
|
||||
it('requires either content or replacements', async () => {
|
||||
const { tool } = makeTool();
|
||||
const result = await tool.call({ key: 'k', summary: 's' } as any, baseContext);
|
||||
expect(result.structured.success).toBe(false);
|
||||
expect(result.markdown).toMatch(/content.*or.*replacements/i);
|
||||
});
|
||||
|
||||
it('updates frontmatter only on an existing page while preserving content', async () => {
|
||||
const { tool, wikiService } = makeTool({
|
||||
wikiService: {
|
||||
readPage: vi.fn().mockResolvedValue({
|
||||
pageKey: 'orbit-customers',
|
||||
frontmatter: {
|
||||
summary: 'Customer source details',
|
||||
usage_mode: 'auto',
|
||||
sort_order: 0,
|
||||
tags: ['notion'],
|
||||
refs: ['notion:old'],
|
||||
sl_refs: ['postgres-warehouse/orbit_analytics.customer'],
|
||||
},
|
||||
content: '# Orbit Customers\n\nSource: Notion - Orbit Customers Source.',
|
||||
}),
|
||||
},
|
||||
});
|
||||
|
||||
const result = await tool.call(
|
||||
{
|
||||
key: 'orbit-customers',
|
||||
summary: 'Customer source details mapped to the warehouse customer view',
|
||||
sl_refs: ['postgres-warehouse/orbit_analytics.customer', 'dbt-main/customer'],
|
||||
} as any,
|
||||
baseContext,
|
||||
);
|
||||
|
||||
expect(result.structured).toMatchObject({ success: true, key: 'orbit-customers', action: 'updated' });
|
||||
expect(wikiService.writePage).toHaveBeenCalledWith(
|
||||
'USER',
|
||||
'u',
|
||||
'orbit-customers',
|
||||
expect.objectContaining({
|
||||
summary: 'Customer source details mapped to the warehouse customer view',
|
||||
tags: ['notion'],
|
||||
refs: ['notion:old'],
|
||||
sl_refs: ['postgres-warehouse/orbit_analytics.customer', 'dbt-main/customer'],
|
||||
}),
|
||||
'# Orbit Customers\n\nSource: Notion - Orbit Customers Source.',
|
||||
expect.any(String),
|
||||
expect.any(String),
|
||||
);
|
||||
});
|
||||
|
||||
it('writes historic-SQL frontmatter fields', async () => {
|
||||
const { tool, wikiService } = makeTool();
|
||||
|
||||
await tool.call(
|
||||
{
|
||||
key: 'monthly-paid-orders',
|
||||
summary: 'Monthly paid orders',
|
||||
tags: ['historic-sql', 'query-pattern'],
|
||||
sl_refs: ['analytics.orders'],
|
||||
source: 'historic-sql',
|
||||
intent: 'Monthly paid order count',
|
||||
tables: ['analytics.orders'],
|
||||
representative_sql: "SELECT count(*) FROM analytics.orders WHERE status = 'paid'",
|
||||
usage: {
|
||||
executions: 42,
|
||||
distinct_users: 3,
|
||||
first_seen: '2026-02-01',
|
||||
last_seen: '2026-05-04',
|
||||
p50_runtime_ms: 100,
|
||||
p95_runtime_ms: 200,
|
||||
error_rate: 0,
|
||||
rows_produced: 42,
|
||||
},
|
||||
fingerprints: ['fp_paid_orders'],
|
||||
content: '## Monthly paid order count',
|
||||
} as any,
|
||||
baseContext,
|
||||
);
|
||||
|
||||
expect(wikiService.writePage.mock.calls[0][3]).toEqual({
|
||||
summary: 'Monthly paid orders',
|
||||
usage_mode: 'auto',
|
||||
sort_order: 0,
|
||||
tags: ['historic-sql', 'query-pattern'],
|
||||
refs: undefined,
|
||||
sl_refs: ['analytics.orders'],
|
||||
source: 'historic-sql',
|
||||
intent: 'Monthly paid order count',
|
||||
tables: ['analytics.orders'],
|
||||
representative_sql: "SELECT count(*) FROM analytics.orders WHERE status = 'paid'",
|
||||
usage: {
|
||||
executions: 42,
|
||||
distinct_users: 3,
|
||||
first_seen: '2026-02-01',
|
||||
last_seen: '2026-05-04',
|
||||
p50_runtime_ms: 100,
|
||||
p95_runtime_ms: 200,
|
||||
error_rate: 0,
|
||||
rows_produced: 42,
|
||||
},
|
||||
fingerprints: ['fp_paid_orders'],
|
||||
});
|
||||
});
|
||||
|
||||
it('preserves historic-SQL frontmatter fields when update omits them', async () => {
|
||||
const existingFrontmatter = {
|
||||
summary: 'Monthly paid orders',
|
||||
usage_mode: 'auto' as const,
|
||||
sort_order: 0,
|
||||
tags: ['historic-sql'],
|
||||
sl_refs: ['analytics.orders'],
|
||||
source: 'historic-sql',
|
||||
intent: 'Monthly paid order count',
|
||||
tables: ['analytics.orders'],
|
||||
representative_sql: "SELECT count(*) FROM analytics.orders WHERE status = 'paid'",
|
||||
usage: {
|
||||
executions: 42,
|
||||
distinct_users: 3,
|
||||
first_seen: '2026-02-01',
|
||||
last_seen: '2026-05-04',
|
||||
p50_runtime_ms: 100,
|
||||
p95_runtime_ms: 200,
|
||||
error_rate: 0,
|
||||
rows_produced: 42,
|
||||
},
|
||||
fingerprints: ['fp_paid_orders'],
|
||||
};
|
||||
const { tool, wikiService } = makeTool({
|
||||
wikiService: {
|
||||
readPage: vi.fn().mockResolvedValue({
|
||||
pageKey: 'monthly-paid-orders',
|
||||
frontmatter: existingFrontmatter,
|
||||
content: 'old body',
|
||||
}),
|
||||
},
|
||||
});
|
||||
|
||||
await tool.call(
|
||||
{
|
||||
key: 'monthly-paid-orders',
|
||||
summary: 'Monthly paid orders updated',
|
||||
content: '## Monthly paid order count updated',
|
||||
} as any,
|
||||
baseContext,
|
||||
);
|
||||
|
||||
expect(wikiService.writePage.mock.calls[0][3]).toEqual({
|
||||
...existingFrontmatter,
|
||||
summary: 'Monthly paid orders updated',
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects frontmatter refs that target missing wiki pages', async () => {
|
||||
const { tool, wikiService } = makeTool({
|
||||
wikiService: {
|
||||
listPageKeys: vi.fn().mockResolvedValue(['orbit-company-overview']),
|
||||
},
|
||||
});
|
||||
|
||||
const result = await tool.call(
|
||||
{
|
||||
key: 'orbit-how-we-work',
|
||||
summary: 'Operating norms',
|
||||
content: '## How We Work',
|
||||
refs: ['orbit-company-overview', 'orbit-team-lanes-detail'],
|
||||
} as any,
|
||||
baseContext,
|
||||
);
|
||||
|
||||
expect(result.structured.success).toBe(false);
|
||||
expect(result.markdown).toMatch(/orbit-team-lanes-detail/);
|
||||
expect(wikiService.writePage).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects inline wiki links that target missing wiki pages', async () => {
|
||||
const { tool, wikiService } = makeTool({
|
||||
wikiService: {
|
||||
listPageKeys: vi.fn().mockResolvedValue(['orbit-company-overview']),
|
||||
},
|
||||
});
|
||||
|
||||
const result = await tool.call(
|
||||
{
|
||||
key: 'orbit-how-we-work',
|
||||
summary: 'Operating norms',
|
||||
content: 'See [[orbit-company-overview]] and [[orbit-team-lanes-detail]].',
|
||||
} as any,
|
||||
baseContext,
|
||||
);
|
||||
|
||||
expect(result.structured.success).toBe(false);
|
||||
expect(result.markdown).toMatch(/orbit-team-lanes-detail/);
|
||||
expect(wikiService.writePage).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('accepts forward refs during ingest sessions for post-pass validation', async () => {
|
||||
const { tool, wikiService } = makeTool({
|
||||
wikiService: {
|
||||
listPageKeys: vi.fn().mockResolvedValue(['orbit-company-overview']),
|
||||
},
|
||||
});
|
||||
const session: ToolSession = {
|
||||
connectionId: 'conn-1',
|
||||
isWorktreeScoped: true,
|
||||
preHead: null,
|
||||
touchedSlSources: createTouchedSlSources(),
|
||||
actions: [],
|
||||
semanticLayerService: {} as any,
|
||||
wikiService: wikiService as any,
|
||||
configService: {} as any,
|
||||
gitService: {} as any,
|
||||
ingest: { runId: 'run-1', jobId: 'job-1', syncId: 'sync-1', sourceKey: 'notion' },
|
||||
};
|
||||
|
||||
const result = await tool.call(
|
||||
{
|
||||
key: 'orbit-how-we-work',
|
||||
summary: 'Operating norms',
|
||||
content: 'See [[orbit-team-lanes-detail]].',
|
||||
refs: ['orbit-company-overview', 'orbit-team-lanes-detail'],
|
||||
} as any,
|
||||
{ ...baseContext, session },
|
||||
);
|
||||
|
||||
expect(result.structured).toMatchObject({ success: true, key: 'orbit-how-we-work', action: 'created' });
|
||||
expect(wikiService.writePage).toHaveBeenCalledTimes(1);
|
||||
expect(session.actions).toContainEqual(
|
||||
expect.objectContaining({ target: 'wiki', type: 'created', key: 'orbit-how-we-work' }),
|
||||
);
|
||||
});
|
||||
});
|
||||
245
packages/cli/src/context/wiki/tools/wiki-write.tool.ts
Normal file
245
packages/cli/src/context/wiki/tools/wiki-write.tool.ts
Normal file
|
|
@ -0,0 +1,245 @@
|
|||
import { z } from 'zod';
|
||||
import type { KnowledgeIndexPort } from '../ports.js';
|
||||
import type { KnowledgeEventPort } from '../ports.js';
|
||||
type BlockScope = 'GLOBAL' | 'USER';
|
||||
import { KnowledgeWikiService } from '../../../context/wiki/knowledge-wiki.service.js';
|
||||
import type { WikiFrontmatter } from '../../../context/wiki/types.js';
|
||||
import { validateFlatWikiKey } from '../keys.js';
|
||||
import { findMissingWikiRefs } from '../wiki-ref-validation.js';
|
||||
import { applySqlEdits } from '../../tools/sql-edit-replacer.js';
|
||||
import { BaseTool, type ToolContext, type ToolOutput } from '../../../context/tools/base-tool.js';
|
||||
import { validateActionRawPaths } from '../../../context/tools/action-raw-paths.js';
|
||||
|
||||
const MAX_USER_BLOCKS = 100;
|
||||
const SYSTEM_AUTHOR = 'System User';
|
||||
const SYSTEM_EMAIL = 'system@example.com';
|
||||
|
||||
const historicSqlUsageFrontmatterSchema = z.object({
|
||||
executions: z.number().int().nonnegative(),
|
||||
distinct_users: z.number().int().nonnegative(),
|
||||
first_seen: z.string().min(1),
|
||||
last_seen: z.string().min(1),
|
||||
p50_runtime_ms: z.number().nonnegative().nullable(),
|
||||
p95_runtime_ms: z.number().nonnegative().nullable(),
|
||||
error_rate: z.number().min(0).max(1),
|
||||
rows_produced: z.number().int().nonnegative().optional(),
|
||||
});
|
||||
|
||||
const wikiWriteInputSchema = z.object({
|
||||
key: z.string().max(120),
|
||||
summary: z.string().max(200),
|
||||
content: z.string().max(4000).optional(),
|
||||
replacements: z
|
||||
.array(z.object({ oldText: z.string(), newText: z.string(), reason: z.string().optional() }))
|
||||
.optional(),
|
||||
tags: z.array(z.string()).optional(),
|
||||
refs: z.array(z.string()).optional(),
|
||||
sl_refs: z.array(z.string()).optional(),
|
||||
source: z.string().optional(),
|
||||
intent: z.string().optional(),
|
||||
tables: z.array(z.string()).optional(),
|
||||
representative_sql: z.string().optional(),
|
||||
usage: historicSqlUsageFrontmatterSchema.optional(),
|
||||
fingerprints: z.array(z.string()).optional(),
|
||||
rawPaths: z
|
||||
.array(z.string().min(1))
|
||||
.optional()
|
||||
.describe('In ingest sessions, raw source file paths that directly support this wiki action.'),
|
||||
});
|
||||
|
||||
type WikiWriteInput = z.infer<typeof wikiWriteInputSchema>;
|
||||
|
||||
interface WikiWriteStructured {
|
||||
success: boolean;
|
||||
key: string;
|
||||
action?: 'created' | 'updated';
|
||||
content?: string;
|
||||
}
|
||||
|
||||
function looksLikeEscapedMarkdown(content: string): boolean {
|
||||
const withoutInlineCode = content.replace(/`[^`]*`/g, '');
|
||||
return /\\n\\n|(?:^|\\n)#{1,6}\s|\\n[-*]\s|\\n\d+\.\s|\\n```|\\n\|/.test(withoutInlineCode);
|
||||
}
|
||||
|
||||
function normalizeAccidentalEscapedMarkdownNewlines(content: string): string {
|
||||
const escapedBreaks = content.match(/\\[rn]/g)?.length ?? 0;
|
||||
if (escapedBreaks < 2) return content;
|
||||
|
||||
const actualBreaks = content.match(/\r?\n/g)?.length ?? 0;
|
||||
if (actualBreaks > 0 && escapedBreaks <= actualBreaks * 4) return content;
|
||||
if (!looksLikeEscapedMarkdown(content)) return content;
|
||||
|
||||
return content.replace(/\\r\\n/g, '\n').replace(/\\n/g, '\n').replace(/\\r/g, '\n');
|
||||
}
|
||||
|
||||
export class WikiWriteTool extends BaseTool<typeof wikiWriteInputSchema> {
|
||||
readonly name = 'wiki_write';
|
||||
|
||||
constructor(
|
||||
private readonly wikiService: KnowledgeWikiService,
|
||||
private readonly pagesRepository: KnowledgeIndexPort,
|
||||
private readonly knowledgeRepository: KnowledgeEventPort,
|
||||
) {
|
||||
super();
|
||||
}
|
||||
|
||||
get description(): string {
|
||||
return `<purpose>
|
||||
Create or update a wiki page. Provide content for create/rewrite, or replacements for targeted edits.
|
||||
For existing pages, you may provide only frontmatter fields such as summary, tags, refs, or sl_refs to update metadata while preserving content.
|
||||
tags/refs/sl_refs use REPLACE semantics: omit to keep existing on update, [] to clear, [values] to set.
|
||||
Keys must be flat file names, not directory paths. Use tags/source frontmatter for grouping.
|
||||
</purpose>`;
|
||||
}
|
||||
|
||||
get inputSchema() {
|
||||
return wikiWriteInputSchema;
|
||||
}
|
||||
|
||||
async call(input: WikiWriteInput, context: ToolContext): Promise<ToolOutput<WikiWriteStructured>> {
|
||||
const wikiService = context.session?.wikiService ?? this.wikiService;
|
||||
const writesGlobal = !!context.session;
|
||||
const skipIndex = context.session?.isWorktreeScoped === true;
|
||||
const keyValidation = validateFlatWikiKey(input.key);
|
||||
if (!keyValidation.ok) {
|
||||
return {
|
||||
markdown: keyValidation.error,
|
||||
structured: { success: false, key: input.key },
|
||||
};
|
||||
}
|
||||
const rawPathValidation = validateActionRawPaths(context.session, input.rawPaths);
|
||||
if (!rawPathValidation.ok) {
|
||||
return {
|
||||
markdown: `Error: ${rawPathValidation.error}`,
|
||||
structured: { success: false, key: input.key },
|
||||
};
|
||||
}
|
||||
|
||||
const scope: BlockScope = writesGlobal ? 'GLOBAL' : 'USER';
|
||||
const scopeId = scope === 'USER' ? context.userId : null;
|
||||
const existing = await wikiService.readPage(scope, scopeId, input.key);
|
||||
|
||||
const content = input.content;
|
||||
const hasContent = typeof content === 'string' && content.length > 0;
|
||||
const hasReplacements = !!input.replacements && input.replacements.length > 0;
|
||||
if (!existing && !hasContent && !hasReplacements) {
|
||||
return {
|
||||
markdown: 'Error: provide either content (for create/rewrite) or replacements (for edits).',
|
||||
structured: { success: false, key: input.key },
|
||||
};
|
||||
}
|
||||
|
||||
if (!existing && !input.content) {
|
||||
return {
|
||||
markdown: `Page "${input.key}" does not exist. Provide content to create it.`,
|
||||
structured: { success: false, key: input.key },
|
||||
};
|
||||
}
|
||||
|
||||
if (scope === 'USER' && !existing) {
|
||||
const count = await this.pagesRepository.getUserPageCount(context.userId);
|
||||
if (count >= MAX_USER_BLOCKS) {
|
||||
return {
|
||||
markdown: `Cannot create "${input.key}": user has reached the limit of ${MAX_USER_BLOCKS} pages.`,
|
||||
structured: { success: false, key: input.key },
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const existingFm = existing?.frontmatter;
|
||||
const resolvedTags = input.tags === undefined ? existingFm?.tags : input.tags;
|
||||
const resolvedRefs = input.refs === undefined ? existingFm?.refs : input.refs;
|
||||
const resolvedSlRefs = input.sl_refs === undefined ? existingFm?.sl_refs : input.sl_refs;
|
||||
|
||||
let finalContent: string;
|
||||
const finalFm: WikiFrontmatter = {
|
||||
summary: input.summary,
|
||||
usage_mode: existingFm?.usage_mode ?? 'auto',
|
||||
sort_order: existingFm?.sort_order ?? 0,
|
||||
tags: resolvedTags,
|
||||
refs: resolvedRefs,
|
||||
sl_refs: resolvedSlRefs,
|
||||
source: input.source === undefined ? existingFm?.source : input.source,
|
||||
intent: input.intent === undefined ? existingFm?.intent : input.intent,
|
||||
tables: input.tables === undefined ? existingFm?.tables : input.tables,
|
||||
representative_sql:
|
||||
input.representative_sql === undefined ? existingFm?.representative_sql : input.representative_sql,
|
||||
usage: input.usage === undefined ? existingFm?.usage : input.usage,
|
||||
fingerprints: input.fingerprints === undefined ? existingFm?.fingerprints : input.fingerprints,
|
||||
};
|
||||
|
||||
if (hasContent) {
|
||||
finalContent = normalizeAccidentalEscapedMarkdownNewlines(content);
|
||||
} else if (hasReplacements) {
|
||||
const editResult = applySqlEdits(existing?.content ?? '', input.replacements ?? []);
|
||||
if (!editResult.success) {
|
||||
return {
|
||||
markdown: `Edit errors: ${editResult.errors.join('; ')}`,
|
||||
structured: { success: false, key: input.key },
|
||||
};
|
||||
}
|
||||
finalContent = editResult.sql;
|
||||
} else {
|
||||
finalContent = existing?.content ?? '';
|
||||
}
|
||||
|
||||
const missingRefs = await findMissingWikiRefs({
|
||||
wikiService,
|
||||
scope,
|
||||
scopeId,
|
||||
pageKey: input.key,
|
||||
refs: finalFm.refs,
|
||||
content: finalContent,
|
||||
});
|
||||
const deferMissingRefs = !!context.session?.ingest;
|
||||
if (!deferMissingRefs && missingRefs.length > 0) {
|
||||
return {
|
||||
markdown:
|
||||
`Error: wiki references target missing page(s): ${missingRefs.join(', ')}. ` +
|
||||
'Create those pages first, retarget the links, or remove the refs.',
|
||||
structured: { success: false, key: input.key },
|
||||
};
|
||||
}
|
||||
|
||||
await wikiService.writePage(scope, scopeId, input.key, finalFm, finalContent, SYSTEM_AUTHOR, SYSTEM_EMAIL);
|
||||
if (!skipIndex) {
|
||||
await wikiService.syncSinglePage(scope, scopeId, input.key, finalFm, finalContent);
|
||||
}
|
||||
|
||||
await this.knowledgeRepository.createEvent({
|
||||
blockId: null,
|
||||
eventType: existing ? 'BLOCK_UPDATED' : 'BLOCK_CREATED',
|
||||
actorId: context.userId,
|
||||
chatId: null,
|
||||
messageId: null,
|
||||
payload: {
|
||||
pageKey: input.key,
|
||||
previousContent: existing ? existing.content.slice(0, 500) : null,
|
||||
},
|
||||
});
|
||||
|
||||
const action = existing ? 'updated' : 'created';
|
||||
if (context.session) {
|
||||
context.session.actions.push({
|
||||
target: 'wiki',
|
||||
type: action,
|
||||
key: input.key,
|
||||
detail: input.summary,
|
||||
...(rawPathValidation.rawPaths ? { rawPaths: rawPathValidation.rawPaths } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
// When the LLM used `replacements` (edit mode), it doesn't have the
|
||||
// post-edit content cached. Returning the result here prevents the
|
||||
// common bug where a follow-up edit uses an oldText string that no
|
||||
// longer matches because a prior edit already changed the page.
|
||||
const markdown = hasReplacements
|
||||
? `Page "${input.key}" ${action}.\n\nCurrent content (use for subsequent edits):\n\n${finalContent}`
|
||||
: `Page "${input.key}" ${action}.`;
|
||||
|
||||
return {
|
||||
markdown,
|
||||
structured: { success: true, key: input.key, action, content: finalContent },
|
||||
};
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue