From 27ba226a570ec0d2f9ed640afc8cb0bf7d0e5fdf Mon Sep 17 00:00:00 2001 From: Andrey Avtomonov Date: Wed, 13 May 2026 18:14:41 +0200 Subject: [PATCH] docs: add unified ingest public CLI surface plan --- ...05-13-unified-ingest-public-cli-surface.md | 1584 +++++++++++++++++ 1 file changed, 1584 insertions(+) create mode 100644 docs/superpowers/plans/2026-05-13-unified-ingest-public-cli-surface.md diff --git a/docs/superpowers/plans/2026-05-13-unified-ingest-public-cli-surface.md b/docs/superpowers/plans/2026-05-13-unified-ingest-public-cli-surface.md new file mode 100644 index 00000000..279d02a8 --- /dev/null +++ b/docs/superpowers/plans/2026-05-13-unified-ingest-public-cli-surface.md @@ -0,0 +1,1584 @@ +# Unified Ingest Public CLI Surface Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make `ktx ingest` the public foreground context-build command for one connection or all configured connections. + +**Architecture:** Reuse the existing `public-ingest.ts` orchestration as the public command engine, then extend it to resolve database depth, query-history intent, warnings, and adapter bypasses for connection-centric ingest. Keep low-level `scan` and adapter-backed `ingest run` available as hidden debug surfaces while normal help, output, generated config, and setup recovery text point to `ktx ingest `. + +**Tech Stack:** TypeScript ESM, Commander, Vitest, KTX CLI/context packages, existing scan and local ingest adapters. + +--- + +## Current audit + +The unified ingest spec is not v1-complete. Relevant implemented pieces exist, +but they are not wired as the public product surface: + +- `packages/cli/src/public-ingest.ts` can plan database connections before + source connections and can call scan or source ingest internals. +- `packages/cli/src/context-build-view.ts` renders a foreground progress view + and captures target progress. +- Historic SQL internals exist in `packages/context/src/ingest/adapters/historic-sql` + and CLI adapter wiring exists in `packages/cli/src/local-adapters.ts`. +- The public CLI still rejects `ktx ingest warehouse`; see + `packages/cli/src/index.test.ts`, test name + `rejects removed public ingest shorthand`. +- Root help still exposes `scan`, `ktx ingest --help` still exposes `run` and + `watch`, and generated default config still includes `live-database`. + +This plan addresses the first v1-blocking slice: the public command surface, +connection-centric execution, public depth flags, query-history run overrides, +hidden legacy debug commands, and stale public wording. Setup depth prompting +and foreground-only state cleanup remain separate v1-blocking work after this +slice. + +## File structure + +- Modify `packages/cli/src/cli-runtime.ts`: add an injectable + `publicIngest` dependency for Commander tests and for command routing. +- Modify `packages/cli/src/commands/ingest-commands.ts`: make the parent + `ktx ingest` command accept `[connectionId]`, `--all`, `--fast`, `--deep`, + `--query-history`, `--no-query-history`, and + `--query-history-window-days`; hide legacy `run` and `watch`. +- Modify `packages/cli/src/commands/scan-commands.ts`: hide `ktx scan` from + root help while keeping direct debug invocation. +- Modify `packages/cli/src/cli-program.ts`: remove `scan` from normal + project-aware root command help expectations only where user-facing. +- Modify `packages/cli/src/public-ingest.ts`: resolve target type, depth, + query-history settings, warnings, readiness failures, and adapter bypasses. +- Modify `packages/cli/src/context-build-view.ts`: rename public labels and + public operation text away from scan terminology. +- Modify `packages/cli/src/ingest.ts`: allow connection-centric ingest to run + an inferred adapter without requiring `ingest.adapters`. +- Modify `packages/cli/src/local-adapters.ts`: accept current-run + query-history overrides for `context.queryHistory` without rewriting config. +- Modify `packages/context/src/project/config.ts`: stop generating + `live-database` and source adapters in default `ktx.yaml`. +- Modify `packages/cli/src/setup-sources.ts`: replace stale recovery command + suggestions with `ktx ingest `. +- Modify `README.md` and script assertions that document normal public command + output. + +## Tasks + +### Task 1: Route the public `ktx ingest` command + +**Files:** +- Modify: `packages/cli/src/cli-runtime.ts` +- Modify: `packages/cli/src/commands/ingest-commands.ts` +- Modify: `packages/cli/src/index.test.ts` +- Modify: `packages/cli/src/dev.test.ts` + +- [ ] **Step 1: Write failing Commander routing tests** + +In `packages/cli/src/index.test.ts`, replace the test named +`rejects removed public ingest shorthand` with: + +```ts + it('routes public connection-centric ingest shorthand', async () => { + const testIo = makeIo(); + const publicIngest = vi.fn().mockResolvedValue(0); + + await expect( + runKtxCli(['--project-dir', '/tmp/project', 'ingest', 'warehouse', '--fast', '--no-input'], testIo.io, { + publicIngest, + }), + ).resolves.toBe(0); + + expect(publicIngest).toHaveBeenCalledWith( + { + command: 'run', + projectDir: '/tmp/project', + targetConnectionId: 'warehouse', + all: false, + json: false, + inputMode: 'disabled', + depth: 'fast', + queryHistory: 'default', + }, + testIo.io, + ); + expect(testIo.stderr()).toBe('Project: /tmp/project\n'); + }); + + it('routes public ingest --all --deep with JSON output', async () => { + const testIo = makeIo(); + const publicIngest = vi.fn().mockResolvedValue(0); + + await expect( + runKtxCli(['--project-dir', '/tmp/project', 'ingest', '--all', '--deep', '--json'], testIo.io, { + publicIngest, + }), + ).resolves.toBe(0); + + expect(publicIngest).toHaveBeenCalledWith( + { + command: 'run', + projectDir: '/tmp/project', + all: true, + json: true, + inputMode: 'auto', + depth: 'deep', + queryHistory: 'default', + }, + testIo.io, + ); + expect(testIo.stderr()).toBe(''); + }); + + it('rejects mutually exclusive public ingest depth flags before dispatch', async () => { + const testIo = makeIo(); + const publicIngest = vi.fn().mockResolvedValue(0); + + await expect( + runKtxCli(['--project-dir', '/tmp/project', 'ingest', 'warehouse', '--fast', '--deep'], testIo.io, { + publicIngest, + }), + ).resolves.toBe(1); + + expect(publicIngest).not.toHaveBeenCalled(); + expect(testIo.stderr()).toContain("option '--deep' cannot be used with option '--fast'"); + }); +``` + +In the existing ingest help test, change the expected help assertions to: + +```ts + expect(testIo.stdout()).toContain('Usage: ktx ingest [options] [connectionId]'); + expect(testIo.stdout()).toContain('Build or inspect KTX context'); + expect(testIo.stdout()).toContain('--all'); + expect(testIo.stdout()).toContain('--fast'); + expect(testIo.stdout()).toContain('--deep'); + expect(testIo.stdout()).toContain('--query-history'); + expect(testIo.stdout()).toContain('--no-query-history'); + expect(testIo.stdout()).toContain('--query-history-window-days '); + expect(testIo.stdout()).toContain('status'); + expect(testIo.stdout()).toContain('replay'); + expect(testIo.stdout()).not.toContain('run'); + expect(testIo.stdout()).not.toContain('watch'); +``` + +In `packages/cli/src/dev.test.ts`, update the generated nested help case for +`['ingest', 'run', '--help']` so it no longer treats legacy run help as a +normal generated public help case. Add this direct hidden-command regression +test instead: + +```ts + it('keeps legacy adapter-backed ingest run callable but hidden from ingest help', async () => { + const helpIo = makeIo(); + const runIo = makeIo(); + const ingest = vi.fn(async () => 0); + + await expect(runKtxCli(['ingest', '--help'], helpIo.io, { ingest })).resolves.toBe(0); + await expect( + runKtxCli( + ['ingest', 'run', '--connection-id', 'warehouse', '--adapter', 'metabase', '--project-dir', '/tmp/project'], + runIo.io, + { ingest }, + ), + ).resolves.toBe(0); + + expect(helpIo.stdout()).not.toContain('run'); + expect(ingest).toHaveBeenCalledWith( + expect.objectContaining({ command: 'run', connectionId: 'warehouse', adapter: 'metabase' }), + runIo.io, + ); + }); +``` + +- [ ] **Step 2: Run the failing Commander tests** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/index.test.ts src/dev.test.ts -t "public connection-centric ingest|public ingest --all|mutually exclusive public ingest|legacy adapter-backed ingest run|prints ingest help" +``` + +Expected: FAIL because `KtxCliDeps` has no `publicIngest`, `ktx ingest +warehouse` is unknown, and `run`/`watch` are still visible in help. + +- [ ] **Step 3: Add the injectable public ingest dependency** + +In `packages/cli/src/cli-runtime.ts`, add this import near the existing CLI +argument type imports: + +```ts +import type { KtxPublicIngestArgs } from './public-ingest.js'; +``` + +In `KtxCliDeps`, add: + +```ts + publicIngest?: (args: KtxPublicIngestArgs, io: KtxCliIo) => Promise; +``` + +- [ ] **Step 4: Register parent `ktx ingest` options and hidden legacy commands** + +In `packages/cli/src/commands/ingest-commands.ts`, add: + +```ts +import type { KtxPublicIngestArgs } from '../public-ingest.js'; +import { parsePositiveIntegerOption } from '../cli-program.js'; +``` + +Replace the current `const ingest = program.command('ingest')...` block with: + +```ts + const ingest = program + .command('ingest') + .description('Build or inspect KTX context') + .argument('[connectionId]', 'Configured connection id to ingest') + .option('--all', 'Ingest all configured connections', false) + .addOption(new Option('--fast', 'Use deterministic database schema ingest').conflicts('deep')) + .addOption(new Option('--deep', 'Use AI-enriched database ingest').conflicts('fast')) + .addOption(new Option('--query-history', 'Include database query-history usage patterns').conflicts('noQueryHistory')) + .addOption(new Option('--no-query-history', 'Skip database query-history usage patterns')) + .option('--query-history-window-days ', 'Query-history lookback window for this run', parsePositiveIntegerOption) + .addOption(new Option('--plain', 'Print plain text output').conflicts(['json'])) + .addOption(new Option('--json', 'Print JSON output').conflicts(['plain'])) + .option('--no-input', 'Disable interactive terminal input') + .showHelpAfterError(); + + ingest.action(async (connectionId: string | undefined, options, command) => { + const { runKtxPublicIngest } = await import('../public-ingest.js'); + const queryHistory = + options.queryHistory === true ? 'enabled' : options.queryHistory === false ? 'disabled' : 'default'; + const args: KtxPublicIngestArgs = { + command: 'run', + projectDir: resolveCommandProjectDir(command), + ...(connectionId ? { targetConnectionId: connectionId } : {}), + all: options.all === true, + json: options.json === true, + inputMode: options.input === false ? 'disabled' : 'auto', + ...(options.fast === true ? { depth: 'fast' as const } : {}), + ...(options.deep === true ? { depth: 'deep' as const } : {}), + queryHistory, + ...(options.queryHistoryWindowDays !== undefined + ? { queryHistoryWindowDays: options.queryHistoryWindowDays } + : {}), + }; + context.setExitCode(await (context.deps.publicIngest ?? runKtxPublicIngest)(args, context.io)); + }); +``` + +Then hide the legacy `run` and `watch` subcommands by changing: + +```ts + .command('run') +``` + +to: + +```ts + .command('run', { hidden: true }) +``` + +and changing: + +```ts + .command('watch') +``` + +to: + +```ts + .command('watch', { hidden: true }) +``` + +- [ ] **Step 5: Run Commander tests again** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/index.test.ts src/dev.test.ts +``` + +Expected: PASS after updating any remaining help text expectations that still +assume public `run` or `watch`. + +- [ ] **Step 6: Commit public route wiring** + +Run: + +```bash +git add packages/cli/src/cli-runtime.ts packages/cli/src/commands/ingest-commands.ts packages/cli/src/index.test.ts packages/cli/src/dev.test.ts +git commit -m "feat(cli): route public connection ingest command" +``` + +### Task 2: Hide top-level `scan` from normal help + +**Files:** +- Modify: `packages/cli/src/commands/scan-commands.ts` +- Modify: `packages/cli/src/index.test.ts` +- Modify: `packages/cli/src/dev.test.ts` +- Modify: `packages/cli/src/cli-program.ts` + +- [ ] **Step 1: Update public help tests** + +In `packages/cli/src/index.test.ts`, in the test `prints the public command +surface in root help`, change the visible command list: + +```ts + for (const command of ['setup', 'connection', 'ingest', 'wiki', 'sl', 'status']) { + expect(testIo.stdout()).toContain(`${command}`); + } + expect(testIo.stdout()).not.toMatch(/^ scan\s/m); +``` + +In `packages/cli/src/dev.test.ts`, keep the direct `['scan', '--help']` case +so the hidden debug command is still callable. + +- [ ] **Step 2: Run the failing scan help tests** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/index.test.ts src/dev.test.ts -t "public command surface|generated nested help" +``` + +Expected: FAIL because root help still prints `scan`. + +- [ ] **Step 3: Hide the scan command** + +In `packages/cli/src/commands/scan-commands.ts`, change: + +```ts + program + .command('scan') +``` + +to: + +```ts + program + .command('scan', { hidden: true }) +``` + +In `packages/cli/src/cli-program.ts`, leave `scan` in +`PROJECT_AWARE_ROOT_COMMANDS` so hidden direct invocations still receive +project-dir behavior: + +```ts +const PROJECT_AWARE_ROOT_COMMANDS = new Set(['setup', 'connection', 'ingest', 'wiki', 'sl', 'status', 'scan']); +``` + +- [ ] **Step 4: Run scan help tests again** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/index.test.ts src/dev.test.ts +``` + +Expected: PASS. + +- [ ] **Step 5: Commit scan help hiding** + +Run: + +```bash +git add packages/cli/src/commands/scan-commands.ts packages/cli/src/index.test.ts packages/cli/src/dev.test.ts packages/cli/src/cli-program.ts +git commit -m "feat(cli): hide standalone scan from public help" +``` + +### Task 3: Resolve public ingest depth, warnings, and query-history intent + +**Files:** +- Modify: `packages/cli/src/public-ingest.ts` +- Modify: `packages/cli/src/public-ingest.test.ts` + +- [ ] **Step 1: Write failing public ingest planner tests** + +In `packages/cli/src/public-ingest.test.ts`, add these tests inside +`describe('buildPublicIngestPlan', ...)`: + +```ts + it('resolves database depth from flags, stored context, and defaults', () => { + const project = projectWithConnections({ + fast_default: { driver: 'postgres' }, + deep_default: { driver: 'postgres', context: { depth: 'deep' } }, + docs: { driver: 'notion' }, + }); + + expect( + buildPublicIngestPlan(project, { + projectDir: '/tmp/project', + targetConnectionId: 'fast_default', + all: false, + queryHistory: 'default', + }).targets[0], + ).toMatchObject({ connectionId: 'fast_default', databaseDepth: 'fast', queryHistory: { enabled: false } }); + + expect( + buildPublicIngestPlan(project, { + projectDir: '/tmp/project', + targetConnectionId: 'deep_default', + all: false, + queryHistory: 'default', + }).targets[0], + ).toMatchObject({ connectionId: 'deep_default', databaseDepth: 'deep' }); + + expect( + buildPublicIngestPlan(project, { + projectDir: '/tmp/project', + targetConnectionId: 'docs', + all: false, + depth: 'deep', + queryHistory: 'default', + }).warnings, + ).toEqual(['--deep affects database ingest only; ignoring it for docs.']); + }); + + it('upgrades effective depth when query history is explicitly enabled', () => { + const project = projectWithConnections({ + warehouse: { driver: 'postgres', context: { queryHistory: { enabled: false } } }, + }); + + const plan = buildPublicIngestPlan(project, { + projectDir: '/tmp/project', + targetConnectionId: 'warehouse', + all: false, + depth: 'fast', + queryHistory: 'enabled', + queryHistoryWindowDays: 30, + }); + + expect(plan.targets[0]).toMatchObject({ + connectionId: 'warehouse', + databaseDepth: 'deep', + queryHistory: { enabled: true, windowDays: 30, dialect: 'postgres' }, + }); + expect(plan.warnings).toEqual(['--query-history requires deep ingest; running warehouse with --deep.']); + }); + + it('warns and skips query history for unsupported database drivers', () => { + const project = projectWithConnections({ local: { driver: 'sqlite' } }); + + const plan = buildPublicIngestPlan(project, { + projectDir: '/tmp/project', + targetConnectionId: 'local', + all: false, + queryHistory: 'enabled', + }); + + expect(plan.targets[0]).toMatchObject({ + connectionId: 'local', + databaseDepth: 'fast', + queryHistory: { enabled: false, unsupported: true }, + }); + expect(plan.warnings).toEqual(['--query-history is not supported for sqlite; running schema ingest for local.']); + }); +``` + +- [ ] **Step 2: Run the failing public ingest planner tests** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/public-ingest.test.ts -t "resolves database depth|upgrades effective depth|unsupported database drivers" +``` + +Expected: FAIL because `depth`, `queryHistory`, `databaseDepth`, and plan +warnings do not exist. + +- [ ] **Step 3: Extend public ingest types** + +In `packages/cli/src/public-ingest.ts`, replace the public step and args types +near the top with: + +```ts +type KtxPublicIngestStepName = 'database-schema' | 'query-history' | 'source-ingest' | 'memory-update'; +type KtxPublicIngestStepStatus = 'done' | 'skipped' | 'failed' | 'not-run'; +type KtxPublicIngestInputMode = 'auto' | 'disabled'; +type KtxPublicIngestDepth = 'fast' | 'deep'; +type KtxPublicIngestQueryHistoryFlag = 'default' | 'enabled' | 'disabled'; +type HistoricSqlDialect = 'postgres' | 'bigquery' | 'snowflake'; +``` + +In the `command: 'run'` variant of `KtxPublicIngestArgs`, add: + +```ts + depth?: KtxPublicIngestDepth; + queryHistory?: KtxPublicIngestQueryHistoryFlag; + queryHistoryWindowDays?: number; +``` + +Replace `KtxPublicIngestPlanTarget` with: + +```ts +export interface KtxPublicIngestPlanTarget { + connectionId: string; + driver: string; + operation: 'database-ingest' | 'source-ingest'; + adapter?: string; + sourceDir?: string; + debugCommand: string; + steps: KtxPublicIngestStepName[]; + databaseDepth?: KtxPublicIngestDepth; + queryHistory?: { + enabled: boolean; + dialect?: HistoricSqlDialect; + windowDays?: number; + unsupported?: boolean; + skippedStoredByFast?: boolean; + }; +} +``` + +Add warnings to `KtxPublicIngestPlan`: + +```ts +export interface KtxPublicIngestPlan { + projectDir: string; + targets: KtxPublicIngestPlanTarget[]; + warnings: string[]; +} +``` + +- [ ] **Step 4: Add depth and query-history resolver helpers** + +Add these helpers after `warehouseDrivers`: + +```ts +const queryHistoryDialectByDriver = new Map([ + ['postgres', 'postgres'], + ['postgresql', 'postgres'], + ['bigquery', 'bigquery'], + ['snowflake', 'snowflake'], +]); + +function connectionContext(connection: KtxProjectConnectionConfig): Record { + const value = connection.context; + return typeof value === 'object' && value !== null && !Array.isArray(value) ? (value as Record) : {}; +} + +function storedDepth(connection: KtxProjectConnectionConfig): KtxPublicIngestDepth | undefined { + const value = connectionContext(connection).depth; + return value === 'fast' || value === 'deep' ? value : undefined; +} + +function storedQueryHistory(connection: KtxProjectConnectionConfig): Record { + const value = connectionContext(connection).queryHistory; + return typeof value === 'object' && value !== null && !Array.isArray(value) ? (value as Record) : {}; +} + +function positiveInteger(value: unknown): number | undefined { + return typeof value === 'number' && Number.isInteger(value) && value > 0 ? value : undefined; +} +``` + +Add: + +```ts +function resolveDatabaseTargetOptions(input: { + connectionId: string; + driver: string; + connection: KtxProjectConnectionConfig; + args: { + depth?: KtxPublicIngestDepth; + queryHistory?: KtxPublicIngestQueryHistoryFlag; + queryHistoryWindowDays?: number; + }; + warnings: string[]; +}): Pick { + const storedQh = storedQueryHistory(input.connection); + const dialect = queryHistoryDialectByDriver.get(input.driver); + const explicitQueryHistory = input.args.queryHistory ?? 'default'; + const storedEnabled = storedQh.enabled === true; + const requestedQh = explicitQueryHistory === 'enabled' || (explicitQueryHistory === 'default' && storedEnabled); + let depth = input.args.depth ?? storedDepth(input.connection) ?? 'fast'; + const queryHistory = { + enabled: false, + ...(input.args.queryHistoryWindowDays !== undefined + ? { windowDays: input.args.queryHistoryWindowDays } + : positiveInteger(storedQh.windowDays) !== undefined + ? { windowDays: positiveInteger(storedQh.windowDays) } + : {}), + }; + + if (requestedQh && !dialect) { + input.warnings.push( + explicitQueryHistory === 'enabled' || input.args.queryHistoryWindowDays !== undefined + ? `--query-history is not supported for ${input.driver}; running schema ingest for ${input.connectionId}.` + : `${input.connectionId} has query history enabled in ktx.yaml, but ${input.driver} does not support it; running schema ingest.`, + ); + return { + databaseDepth: depth, + queryHistory: { ...queryHistory, unsupported: true }, + steps: ['database-schema'], + }; + } + + if (requestedQh && dialect) { + if (depth === 'fast') { + input.warnings.push(`--query-history requires deep ingest; running ${input.connectionId} with --deep.`); + } + depth = 'deep'; + return { + databaseDepth: depth, + queryHistory: { ...queryHistory, enabled: true, dialect }, + steps: ['database-schema', 'query-history'], + }; + } + + if (input.args.depth === 'fast' && explicitQueryHistory !== 'enabled' && storedEnabled) { + input.warnings.push( + `${input.connectionId} has query history enabled in ktx.yaml, but --fast skips query-history processing.`, + ); + return { + databaseDepth: 'fast', + queryHistory: { ...queryHistory, skippedStoredByFast: true }, + steps: ['database-schema'], + }; + } + + return { + databaseDepth: depth, + queryHistory, + steps: ['database-schema'], + }; +} +``` + +- [ ] **Step 5: Use the resolver in plan construction** + +Change `targetForConnection` to accept args and warnings: + +```ts +function targetForConnection( + connectionId: string, + connection: KtxProjectConnectionConfig, + args: { + depth?: KtxPublicIngestDepth; + queryHistory?: KtxPublicIngestQueryHistoryFlag; + queryHistoryWindowDays?: number; + }, + warnings: string[], +): KtxPublicIngestPlanTarget { +``` + +In the source-adapter branch, before returning, add: + +```ts + if (args.depth) { + warnings.push(`--${args.depth} affects database ingest only; ignoring it for ${connectionId}.`); + } + if (args.queryHistory === 'enabled' || args.queryHistoryWindowDays !== undefined) { + warnings.push(`--query-history affects database ingest only; ignoring it for ${connectionId}.`); + } +``` + +Change the source debug command to: + +```ts + debugCommand: `ktx ingest ${connectionId} --debug`, +``` + +In the warehouse branch, replace the return object with: + +```ts + const options = resolveDatabaseTargetOptions({ connectionId, driver, connection, args, warnings }); + return { + connectionId, + driver, + operation: 'database-ingest', + debugCommand: `ktx ingest ${connectionId} --debug`, + ...options, + }; +``` + +In `buildPublicIngestPlan`, add warnings and return them: + +```ts + const warnings: string[] = []; + const targets = selected.map(([connectionId, connection]) => targetForConnection(connectionId, connection, args, warnings)); + return { + projectDir: args.projectDir, + targets: [ + ...targets.filter((t) => t.operation === 'database-ingest'), + ...targets.filter((t) => t.operation === 'source-ingest'), + ], + warnings, + }; +``` + +- [ ] **Step 6: Run planner tests again** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/public-ingest.test.ts -t "buildPublicIngestPlan" +``` + +Expected: PASS after updating older expected target snapshots from +`operation: 'scan'` to `operation: 'database-ingest'`, from `steps: ['scan']` +to `steps: ['database-schema']`, and adding `warnings: []`. + +- [ ] **Step 7: Commit public ingest planning** + +Run: + +```bash +git add packages/cli/src/public-ingest.ts packages/cli/src/public-ingest.test.ts +git commit -m "feat(cli): plan public ingest depth and query history" +``` + +### Task 4: Execute database depth and query-history facets + +**Files:** +- Modify: `packages/cli/src/public-ingest.ts` +- Modify: `packages/cli/src/public-ingest.test.ts` +- Modify: `packages/cli/src/ingest.ts` + +- [ ] **Step 1: Write failing execution tests** + +In `packages/cli/src/public-ingest.test.ts`, add: + +```ts + it('maps fast and deep database targets to scan internals', async () => { + const io = makeIo(); + const project = projectWithConnections({ + fast: { driver: 'postgres' }, + deep: { driver: 'postgres', context: { depth: 'deep' } }, + }); + const runScan = vi.fn(async () => 0); + + await expect( + runKtxPublicIngest( + { command: 'run', projectDir: '/tmp/project', all: true, json: false, inputMode: 'disabled', queryHistory: 'default' }, + io.io, + { loadProject: vi.fn(async () => project), runScan }, + ), + ).resolves.toBe(0); + + expect(runScan).toHaveBeenNthCalledWith( + 1, + expect.objectContaining({ connectionId: 'deep', mode: 'enriched', detectRelationships: true }), + expect.anything(), + ); + expect(runScan).toHaveBeenNthCalledWith( + 2, + expect.objectContaining({ connectionId: 'fast', mode: 'structural', detectRelationships: false }), + expect.anything(), + ); + }); + + it('runs query history after schema ingest with current-run window override', async () => { + const io = makeIo(); + const project = projectWithConnections({ + warehouse: { driver: 'postgres', context: { queryHistory: { enabled: true, windowDays: 90 } } }, + }); + const runScan = vi.fn(async () => 0); + const runIngest = vi.fn(async () => 0); + + await expect( + runKtxPublicIngest( + { + command: 'run', + projectDir: '/tmp/project', + targetConnectionId: 'warehouse', + all: false, + json: false, + inputMode: 'disabled', + queryHistory: 'enabled', + queryHistoryWindowDays: 30, + }, + io.io, + { loadProject: vi.fn(async () => project), runScan, runIngest }, + ), + ).resolves.toBe(0); + + expect(runScan).toHaveBeenCalledWith( + expect.objectContaining({ connectionId: 'warehouse', mode: 'enriched' }), + expect.anything(), + ); + expect(runIngest).toHaveBeenCalledWith( + expect.objectContaining({ + command: 'run', + connectionId: 'warehouse', + adapter: 'historic-sql', + allowImplicitAdapter: true, + historicSqlPullConfigOverride: expect.objectContaining({ dialect: 'postgres', windowDays: 30 }), + }), + expect.anything(), + ); + }); +``` + +- [ ] **Step 2: Run the failing execution tests** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/public-ingest.test.ts -t "maps fast and deep|runs query history" +``` + +Expected: FAIL because execution still uses `scanMode`, no query-history step +exists, and `KtxIngestArgs` has no implicit-adapter fields. + +- [ ] **Step 3: Add implicit adapter and query-history override fields** + +In `packages/cli/src/ingest.ts`, extend the `command: 'run'` args type: + +```ts + allowImplicitAdapter?: boolean; + historicSqlPullConfigOverride?: Record; +``` + +In the `adapterOptions` object inside `runKtxIngest`, add: + +```ts + ...(args.historicSqlPullConfigOverride + ? { historicSqlPullConfigOverride: args.historicSqlPullConfigOverride } + : {}), +``` + +Before calling `executeLocalIngest`, create the project used for local ingest: + +```ts + const ingestProject = + args.allowImplicitAdapter && !project.config.ingest.adapters.includes(args.adapter) + ? { + ...project, + config: { + ...project.config, + ingest: { + ...project.config.ingest, + adapters: [...project.config.ingest.adapters, args.adapter], + }, + }, + } + : project; +``` + +Then pass `ingestProject` instead of `project` to `runLocalMetabaseIngest`, +`createAdapters`, `createQueryExecutor`, and `executeLocalIngest` in the +`command: 'run'` branch. + +Keep `packages/context/src/ingest/local-ingest.ts` unchanged. The public path +satisfies its strict `assertConfigured()` contract by passing an in-memory +project config whose adapter list includes the inferred adapter for this run. + +- [ ] **Step 4: Execute database targets from effective depth** + +In `packages/cli/src/public-ingest.ts`, update the database branch of +`executePublicIngestTarget`: + +```ts + if (target.operation === 'database-ingest') { + const { runKtxScan } = await import('./scan.js'); + const scanArgs: KtxScanArgs = { + command: 'run', + projectDir: args.projectDir, + connectionId: target.connectionId, + mode: target.databaseDepth === 'deep' ? 'enriched' : 'structural', + detectRelationships: target.databaseDepth === 'deep' ? true : false, + dryRun: false, + }; + const runScan = deps.runScan ?? runKtxScan; + const scanExitCode = deps.scanProgress + ? await runScan(scanArgs, io, { progress: deps.scanProgress }) + : await runScan(scanArgs, io); + if (scanExitCode !== 0) { + return markTargetResult(target, 'failed', 'database-schema'); + } + + if (target.queryHistory?.enabled === true) { + const { runKtxIngest } = await import('./ingest.js'); + const runIngest = deps.runIngest ?? runKtxIngest; + const ingestArgs: KtxIngestArgs = { + command: 'run', + projectDir: args.projectDir, + connectionId: target.connectionId, + adapter: 'historic-sql', + outputMode: sourceIngestOutputMode(args, io), + inputMode: args.inputMode, + allowImplicitAdapter: true, + historicSqlPullConfigOverride: { + dialect: target.queryHistory.dialect, + ...(target.queryHistory.windowDays !== undefined ? { windowDays: target.queryHistory.windowDays } : {}), + }, + }; + const qhExitCode = await runIngest(ingestArgs, io); + if (qhExitCode !== 0) { + return markTargetResult(target, 'failed', 'query-history'); + } + } + + return markTargetResult(target, 'done'); + } +``` + +Update `markTargetResult` to accept the failed operation: + +```ts +function markTargetResult( + target: KtxPublicIngestPlanTarget, + status: 'done' | 'failed', + failedOperation?: KtxPublicIngestStepName, +): KtxPublicIngestTargetResult { +``` + +Inside the function, replace the failed-operation selection with: + +```ts + const selectedFailedOperation = + failedOperation ?? (target.operation === 'database-ingest' ? 'database-schema' : 'source-ingest'); +``` + +Then use `selectedFailedOperation` in the failed-step comparison and detail. + +- [ ] **Step 5: Print plan warnings before results** + +In `runKtxPublicIngest`, after building `plan` and before executing targets, +add: + +```ts + if (!args.json && plan.warnings.length > 0) { + for (const warning of plan.warnings) { + io.stderr.write(`Warning: ${warning}\n`); + } + } +``` + +For JSON output, the existing `{ plan, results }` payload now includes +`plan.warnings`. + +- [ ] **Step 6: Run execution tests again** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/public-ingest.test.ts src/ingest.test.ts +``` + +Expected: PASS after updating public result table labels from `Scan` to +`Database` or `Schema` in existing assertions. + +- [ ] **Step 7: Commit public execution behavior** + +Run: + +```bash +git add packages/cli/src/public-ingest.ts packages/cli/src/public-ingest.test.ts packages/cli/src/ingest.ts +git commit -m "feat(cli): execute public database ingest facets" +``` + +### Task 5: Accept `context.queryHistory` in historic-SQL adapter plumbing + +**Files:** +- Modify: `packages/cli/src/local-adapters.ts` +- Modify: `packages/cli/src/local-adapters.test.ts` +- Modify: `packages/context/src/ingest/local-adapters.ts` +- Modify: `packages/context/src/ingest/local-adapters.test.ts` + +- [ ] **Step 1: Write failing query-history config tests** + +In `packages/context/src/ingest/local-adapters.test.ts`, add: + +```ts + it('maps connection context.queryHistory to historic-sql pull config', async () => { + const project = projectWithConnections({ + warehouse: { + driver: 'postgres', + context: { + queryHistory: { + enabled: true, + windowDays: 45, + minExecutions: 7, + filters: { dropTrivialProbes: true }, + }, + }, + }, + }); + const adapter = { source: 'historic-sql' } as never; + + await expect(localPullConfigForAdapter(project, adapter, 'warehouse')).resolves.toMatchObject({ + dialect: 'postgres', + windowDays: 45, + minExecutions: 7, + filters: { dropTrivialProbes: true }, + }); + }); + + it('prefers context.queryHistory over legacy historicSql', async () => { + const project = projectWithConnections({ + warehouse: { + driver: 'postgres', + historicSql: { enabled: true, dialect: 'postgres', windowDays: 90 }, + context: { queryHistory: { enabled: true, windowDays: 30 } }, + }, + }); + const adapter = { source: 'historic-sql' } as never; + + await expect(localPullConfigForAdapter(project, adapter, 'warehouse')).resolves.toMatchObject({ + dialect: 'postgres', + windowDays: 30, + }); + }); +``` + +In `packages/cli/src/local-adapters.test.ts`, add a test that creates a +Postgres connection with `context.queryHistory.enabled: true`, calls +`createKtxCliLocalIngestAdapters(project, { historicSqlConnectionId: +'warehouse' })`, and expects one adapter with `source === 'historic-sql'`. + +- [ ] **Step 2: Run the failing adapter tests** + +Run: + +```bash +pnpm --filter @ktx/context exec vitest run src/ingest/local-adapters.test.ts +pnpm --filter @ktx/cli exec vitest run src/local-adapters.test.ts +``` + +Expected: FAIL because both layers only look at `connection.historicSql`. + +- [ ] **Step 3: Add context-query-history mapping in context local adapters** + +In `packages/context/src/ingest/local-adapters.ts`, add: + +```ts +const historicSqlDialectByDriver = new Map([ + ['postgres', 'postgres'], + ['postgresql', 'postgres'], + ['bigquery', 'bigquery'], + ['snowflake', 'snowflake'], +]); + +function queryHistoryRecord(connection: unknown): Record | null { + if (!isRecord(connection)) return null; + const context = isRecord(connection.context) ? connection.context : null; + const queryHistory = isRecord(context?.queryHistory) ? context.queryHistory : null; + return queryHistory; +} + +function queryHistoryPullConfig(connection: unknown): Record | null { + const queryHistory = queryHistoryRecord(connection); + if (queryHistory?.enabled !== true || !isRecord(connection)) return null; + const dialect = historicSqlDialectByDriver.get(String(connection.driver ?? '').toLowerCase()); + if (!dialect) return null; + return { ...queryHistory, dialect }; +} +``` + +In `localPullConfigForAdapter`, replace the historic-SQL block with: + +```ts + if (adapter.source === HISTORIC_SQL_SOURCE_KEY) { + const queryHistory = queryHistoryPullConfig(connection); + if (queryHistory) { + return historicSqlUnifiedPullConfigSchema.parse(queryHistory); + } + const historicSql = isRecord(connection?.historicSql) ? connection.historicSql : null; + if (historicSql?.enabled !== true) { + throw new Error(`Connection "${connectionId}" does not have context.queryHistory.enabled: true`); + } + return historicSqlUnifiedPullConfigSchema.parse({ + ...historicSql, + }); + } +``` + +- [ ] **Step 4: Add context-query-history detection in CLI local adapters** + +In `packages/cli/src/local-adapters.ts`, replace `enabledHistoricSqlDialect` +with: + +```ts +function enabledHistoricSqlDialect(connection: unknown): 'postgres' | 'bigquery' | 'snowflake' | null { + const direct = historicSqlRecord(connection); + const context = + connection && typeof connection === 'object' && !Array.isArray(connection) + ? (connection as { context?: unknown }).context + : null; + const queryHistory = + context && typeof context === 'object' && !Array.isArray(context) + ? (context as { queryHistory?: unknown }).queryHistory + : null; + const enabled = + queryHistory && typeof queryHistory === 'object' && !Array.isArray(queryHistory) + ? (queryHistory as { enabled?: unknown }).enabled === true + : direct?.enabled === true; + if (!enabled) { + return null; + } + const driver = String((connection as { driver?: unknown })?.driver ?? '').toLowerCase(); + if (driver === 'postgres' || driver === 'postgresql') return 'postgres'; + if (driver === 'bigquery') return 'bigquery'; + if (driver === 'snowflake') return 'snowflake'; + const legacyDialect = String(direct?.dialect ?? '').toLowerCase(); + return legacyDialect === 'postgres' || legacyDialect === 'bigquery' || legacyDialect === 'snowflake' + ? legacyDialect + : null; +} +``` + +- [ ] **Step 5: Run adapter tests again** + +Run: + +```bash +pnpm --filter @ktx/context exec vitest run src/ingest/local-adapters.test.ts +pnpm --filter @ktx/cli exec vitest run src/local-adapters.test.ts +``` + +Expected: PASS. + +- [ ] **Step 6: Commit query-history adapter config** + +Run: + +```bash +git add packages/context/src/ingest/local-adapters.ts packages/context/src/ingest/local-adapters.test.ts packages/cli/src/local-adapters.ts packages/cli/src/local-adapters.test.ts +git commit -m "feat(ingest): read connection query history config" +``` + +### Task 6: Remove normal `live-database`, adapter, and scan wording from public output + +**Files:** +- Modify: `packages/cli/src/public-ingest.ts` +- Modify: `packages/cli/src/context-build-view.ts` +- Modify: `packages/cli/src/context-build-view.test.ts` +- Modify: `packages/cli/src/setup-sources.ts` +- Modify: `packages/cli/src/setup-sources.test.ts` + +- [ ] **Step 1: Write failing wording tests** + +In `packages/cli/src/context-build-view.test.ts`, change the group label +assertions from `Primary sources:` to `Databases:` and update the running +database detail test to expect `reading schema` instead of `scanning...`. + +Add this setup recovery assertion in the test covering failed initial source +ingest in `packages/cli/src/setup-sources.test.ts`: + +```ts + expect(io.stdout()).toContain(`Run later: ktx ingest ${connectionId}`); + expect(io.stdout()).not.toContain('ktx ingest run --connection-id'); + expect(io.stdout()).not.toContain('--adapter'); +``` + +- [ ] **Step 2: Run failing wording tests** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/context-build-view.test.ts src/setup-sources.test.ts -t "Databases|reading schema|Run later" +``` + +Expected: FAIL because labels still say `Primary sources`, running database +detail says `scanning...`, and setup recovery still suggests adapter-backed +ingest. + +- [ ] **Step 3: Update public render labels** + +In `packages/cli/src/context-build-view.ts`, change: + +```ts + ...renderTargetGroup('Primary sources', state.primarySources, state.frame, styled, width), +``` + +to: + +```ts + ...renderTargetGroup('Databases', state.primarySources, state.frame, styled, width), +``` + +In `targetDetail`, change: + +```ts + ?? (target.target.operation === 'scan' ? 'scanning...' : 'ingesting...'); +``` + +to: + +```ts + ?? (target.target.operation === 'database-ingest' ? 'reading schema' : 'ingesting...'); +``` + +Update type comparisons in this file from `'scan'` to `'database-ingest'` for +public target operation checks. + +- [ ] **Step 4: Update setup source recovery text** + +In `packages/cli/src/setup-sources.ts`, replace: + +```ts + input.io.stdout.write(`│ Run later: ktx ingest run --connection-id ${input.connectionId} --adapter \n`); +``` + +with: + +```ts + input.io.stdout.write(`│ Run later: ktx ingest ${input.connectionId}\n`); +``` + +- [ ] **Step 5: Run wording tests again** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/context-build-view.test.ts src/setup-sources.test.ts +``` + +Expected: PASS after updating existing snapshots for the new public operation +name. + +- [ ] **Step 6: Commit public wording cleanup** + +Run: + +```bash +git add packages/cli/src/public-ingest.ts packages/cli/src/context-build-view.ts packages/cli/src/context-build-view.test.ts packages/cli/src/setup-sources.ts packages/cli/src/setup-sources.test.ts +git commit -m "fix(cli): use public ingest wording" +``` + +### Task 7: Stop generating adapter allow-list entries in normal config + +**Files:** +- Modify: `packages/context/src/project/config.ts` +- Modify: `packages/context/src/project/config.test.ts` +- Modify: `packages/cli/src/setup-sources.ts` +- Modify: `packages/cli/src/setup-sources.test.ts` +- Modify: `packages/cli/src/setup-databases.ts` +- Modify: `packages/cli/src/setup-databases.test.ts` + +- [ ] **Step 1: Write failing config tests** + +In `packages/context/src/project/config.test.ts`, update default assertions: + +```ts + ingest: { + adapters: [], +``` + +and: + +```ts + expect(serialized).not.toContain('live-database'); + expect(parsed.ingest.adapters).toEqual([]); +``` + +In setup database and source tests, add assertions after generated config is +read: + +```ts + expect(configText).not.toContain('live-database'); + expect(configText).not.toContain('historic-sql'); + expect(configText).not.toMatch(/^\s+adapters:/m); +``` + +- [ ] **Step 2: Run failing config tests** + +Run: + +```bash +pnpm --filter @ktx/context exec vitest run src/project/config.test.ts +pnpm --filter @ktx/cli exec vitest run src/setup-databases.test.ts src/setup-sources.test.ts +``` + +Expected: FAIL because defaults and setup still write adapter entries. + +- [ ] **Step 3: Change default config** + +In `packages/context/src/project/config.ts`, change: + +```ts + adapters: ['live-database', 'lookml', 'metabase', 'metricflow', 'notion'], +``` + +to: + +```ts + adapters: [], +``` + +- [ ] **Step 4: Stop setup from appending normal source adapters** + +In `packages/cli/src/setup-sources.ts`, change `writeSourceConnection` so the +new config only writes `connections`: + +```ts + const nextConfig = { + ...project.config, + connections: { + ...project.config.connections, + [connectionId]: connection, + }, + }; +``` + +Remove the `adapters` mutation in that helper and remove adapter rollback code +that only exists to undo automatic adapter appends. + +- [ ] **Step 5: Stop Historic SQL setup from appending adapters** + +In `packages/cli/src/setup-databases.ts`, change `ensureHistoricSqlIngestDefaults` +so it only raises `ingest.workUnits.maxConcurrency`: + +```ts +async function ensureHistoricSqlIngestDefaults(projectDir: string): Promise { + const project = await loadKtxProject({ projectDir }); + const maxConcurrency = Math.max( + project.config.ingest.workUnits.maxConcurrency, + HISTORIC_SQL_WORK_UNIT_MAX_CONCURRENCY, + ); + if (maxConcurrency === project.config.ingest.workUnits.maxConcurrency) { + return; + } + await writeFile( + project.configPath, + serializeKtxProjectConfig({ + ...project.config, + ingest: { + ...project.config.ingest, + workUnits: { + ...project.config.ingest.workUnits, + maxConcurrency, + }, + }, + }), + 'utf-8', + ); +} +``` + +- [ ] **Step 6: Run config tests again** + +Run: + +```bash +pnpm --filter @ktx/context exec vitest run src/project/config.test.ts +pnpm --filter @ktx/cli exec vitest run src/setup-databases.test.ts src/setup-sources.test.ts src/public-ingest.test.ts +``` + +Expected: PASS. Public source ingest still works because Task 4 synthesizes the +inferred adapter for public connection-centric runs. + +- [ ] **Step 7: Commit config cleanup** + +Run: + +```bash +git add packages/context/src/project/config.ts packages/context/src/project/config.test.ts packages/cli/src/setup-sources.ts packages/cli/src/setup-sources.test.ts packages/cli/src/setup-databases.ts packages/cli/src/setup-databases.test.ts +git commit -m "fix(config): stop generating ingest adapter allow lists" +``` + +### Task 8: Update public docs and script assertions + +**Files:** +- Modify: `README.md` +- Modify: `scripts/examples-docs.test.mjs` +- Modify: `scripts/package-artifacts.mjs` +- Modify: `scripts/package-artifacts.test.mjs` +- Modify: `scripts/installed-live-database-smoke.mjs` +- Modify: `scripts/installed-live-database-smoke.test.mjs` + +- [ ] **Step 1: Write failing docs assertion changes** + +In `scripts/examples-docs.test.mjs`, replace assertions that require +`ktx scan `, `ktx scan [options]`, and +`live-database/` in normal README output with assertions for: + +```js +assert.match(buildingContext, /ktx ingest /); +assert.match(buildingContext, /ktx ingest --all/); +assert.doesNotMatch(rootReadme, /live-database\//); +assert.doesNotMatch(rootReadme, /ktx scan/); +``` + +In package artifact smoke tests, change normal public smoke labels from +`ktx scan structural` and `ktx scan enriched` to `ktx ingest fast` and +`ktx ingest deep`. + +- [ ] **Step 2: Run failing docs/script tests** + +Run: + +```bash +node --test scripts/examples-docs.test.mjs scripts/package-artifacts.test.mjs scripts/installed-live-database-smoke.test.mjs +``` + +Expected: FAIL because docs and smoke scripts still mention `scan` and +`live-database`. + +- [ ] **Step 3: Update README public examples** + +In `README.md`, replace normal context-build examples: + +```md +ktx scan warehouse --project-dir "$PROJECT_DIR" +``` + +with: + +```md +ktx ingest warehouse --project-dir "$PROJECT_DIR" --fast +``` + +Replace enriched examples with: + +```md +ktx ingest warehouse --project-dir "$PROJECT_DIR" --deep +``` + +Replace adapter-backed ingest examples for normal users with: + +```md +ktx ingest notion --project-dir "$PROJECT_DIR" +``` + +Keep internal artifact paths only in sections explicitly labeled as debug or +implementation details. + +- [ ] **Step 4: Update smoke scripts to use public ingest** + +In `scripts/package-artifacts.mjs`, replace public scan smoke invocations with: + +```js +const structuralScan = await run('pnpm', [ + 'exec', + 'ktx', + 'ingest', + 'warehouse', + '--project-dir', + projectDir, + '--fast', + '--no-input', +]); +``` + +and: + +```js +const enrichedScan = await run('pnpm', [ + 'exec', + 'ktx', + 'ingest', + 'warehouse', + '--project-dir', + projectDir, + '--deep', + '--no-input', +]); +``` + +Update expected output matches from `Mode: structural` and `Mode: enriched` to +the public result summary that `runKtxPublicIngest` prints, for example +`Database schema` or `database-schema done` depending on the final Task 4 +rendering. + +In `scripts/installed-live-database-smoke.mjs`, keep the file name if renaming +would churn scripts, but change the public CLI invocation from adapter-backed +`ktx ingest run --adapter live-database` to: + +```js +return ['exec', 'ktx', 'ingest', connectionId, '--project-dir', projectDir, '--fast', '--no-input']; +``` + +- [ ] **Step 5: Run docs/script tests again** + +Run: + +```bash +node --test scripts/examples-docs.test.mjs scripts/package-artifacts.test.mjs scripts/installed-live-database-smoke.test.mjs +``` + +Expected: PASS. + +- [ ] **Step 6: Commit docs and smoke cleanup** + +Run: + +```bash +git add README.md scripts/examples-docs.test.mjs scripts/package-artifacts.mjs scripts/package-artifacts.test.mjs scripts/installed-live-database-smoke.mjs scripts/installed-live-database-smoke.test.mjs +git commit -m "docs: document public ingest command" +``` + +### Task 9: Run final verification + +**Files:** +- Verify only. + +- [ ] **Step 1: Run focused CLI and context tests** + +Run: + +```bash +pnpm --filter @ktx/cli exec vitest run src/index.test.ts src/dev.test.ts src/public-ingest.test.ts src/context-build-view.test.ts src/ingest.test.ts src/local-adapters.test.ts src/setup-sources.test.ts src/setup-databases.test.ts +pnpm --filter @ktx/context exec vitest run src/project/config.test.ts src/ingest/local-adapters.test.ts +``` + +Expected: PASS. + +- [ ] **Step 2: Run workspace type checks for touched packages** + +Run: + +```bash +pnpm --filter @ktx/cli run type-check +pnpm --filter @ktx/context run type-check +``` + +Expected: PASS. + +- [ ] **Step 3: Run docs and script tests** + +Run: + +```bash +node --test scripts/examples-docs.test.mjs scripts/package-artifacts.test.mjs scripts/installed-live-database-smoke.test.mjs +``` + +Expected: PASS. + +- [ ] **Step 4: Run dead-code check** + +Run: + +```bash +pnpm run dead-code +``` + +Expected: PASS, or only pre-existing findings unrelated to the files changed +in this plan. + +- [ ] **Step 5: Commit any verification-only expectation fixes** + +If verification required expectation-only changes, run: + +```bash +git add packages/cli/src packages/context/src scripts README.md +git commit -m "test: align ingest surface expectations" +``` + +If there were no changes, do not create an empty commit. + +## Self-review notes + +Spec coverage in this plan: + +- Covers `ktx ingest ` and `ktx ingest --all`. +- Covers public `--fast` and `--deep` mapping to structural and enriched scan + internals. +- Covers hidden legacy `scan`, `ingest run`, and `ingest watch` help behavior. +- Covers adapter allow-list bypass for public connection-centric ingest. +- Covers current-run query-history enablement and window override. +- Covers normal generated config removing adapter allow lists. +- Covers normal help, docs, setup recovery text, and progress wording. + +Known v1-blocking work not included in this plan: + +- Setup must ask for and store `connections..context.depth`. +- Setup readiness must treat fast and deep contexts differently. +- Setup context state must remove detach, watch, resume, stop, paused, and + background subprocess behavior. +- Config rewrite must migrate legacy `connection.historicSql` into + `connection.context.queryHistory`. +- Config/setup validation must reject connection ids that collide with + surviving ingest subcommands. + +Placeholder scan: no task uses deferred code markers or unnamed edge handling. +Each implementation task names exact files, tests, commands, and the concrete +code shape to add.