mirror of
https://github.com/Kaelio/ktx.git
synced 2026-06-28 08:49:38 +02:00
feat: merge ingest and scan
* docs: add CLI component reuse guidance * docs: add unified ingest ux design * Refine unified ingest UX design after adversarial review iteration 1 * Refine unified ingest UX design after adversarial review iteration 2 * Refine unified ingest UX design after adversarial review iteration 3 * feat(cli): route public connection ingest command * feat(cli): hide standalone scan from public help * feat(cli): plan public ingest depth and query history * feat(cli): execute public database ingest facets * feat(ingest): read connection query history config * fix(cli): use public ingest wording * fix(config): stop generating ingest adapter allow lists * docs: document public ingest command * test: align ingest surface expectations * docs: add unified ingest public CLI surface plan * feat(cli): preflight deep public ingest readiness * feat(setup): store query history in connection context * feat(setup): store database context depth * feat(setup): verify context readiness by database depth * fix(setup): keep context build foreground only * fix(config): reject reserved ingest connection ids * test: close unified ingest v1 expectations * docs: add unified ingest v1 closure plan * fix(ingest): bypass adapter allow-list for public source ingest * fix(ingest): honor query history window intent * fix(ingest): hide scan internals from public database ingest * feat(ingest): use foreground view for interactive public ingest * fix(setup): use schema context and query history wording * test(cli): verify unified ingest public output * docs: add unified ingest v1 public output closure plan * fix(setup): forward query history flags * fix(setup): prompt for postgres query history * fix(status): report query history readiness * fix(ingest): remove legacy public guidance * fix(ingest): polish foreground retry copy * docs(examples): use unified query history wording * chore(ingest): finish public query history cleanup * docs: add unified ingest v1 query history status cleanup plan * test(docs): cover unified ingest public docs * docs: align ingest CLI reference with unified UX * docs: update context build guides for unified ingest * docs: update setup and primary source ingest wording * docs: stop advertising adapter-backed example ingest * docs: close unified ingest public docs gaps * docs: add unified ingest v1 docs site closure plan * fix: render unified ingest foreground warnings * fix: explain query history schema order * fix: add public ingest retry guidance * fix: align setup next steps with unified ingest * fix: remove scan wording from demo progress * test: verify unified ingest ux closure * docs: add unified ingest v1 foreground and retry closure plan * fix(cli): preserve query-history pull config in public ingest * fix(cli): omit hidden commands from docs command tree * test(cli): close unified ingest final public surface checks * docs: add unified ingest v1 final public surface closure plan * fix(cli): use public source labels in ingest reports * fix(cli): suppress low-level public ingest output * test(cli): verify unified ingest public plain output * docs: add unified ingest v1 public plain output closure plan * fix(cli): add public ingest copy sanitizers * fix(cli): sanitize public ingest progress copy * fix(cli): rename setup schema scope prompt * docs(plan): add progress copy closure; test: align setup back-nav fixture Adds the iter9 plan and updates the setup back-navigation test fixture to pass disableQueryHistory plus listSchemas/listTables stubs that the unified ingest setup step now requires. * docs(plan): add final ux labels plan with narrowed label scans * fix(cli): aggregate unsupported query-history warnings * fix(cli): align setup database labels * test(cli): fix setup database test type-check * fix(cli): remove primary-source wording from setup output * test(cli): verify unified ingest setup closure * docs(plan): add unified ingest v1 verification copy closure plan * fix(cli): remove top-level scan command * fix(cli): remove legacy ingest and wiki commands * Merge scan into ingest flow * feat(cli): split ingest progress into per-phase rows, rename work units to tasks Each database target in the unified ingest dashboard now renders one row per real subprocess (Schema, then Query history when enabled) instead of a single combined bar. Each phase has its own monotonic 0-100% bar so the progress never snaps back to zero when historic-sql starts after scan completes. Completed phases keep their final bar, summary, and elapsed time visible as an inline audit trail; queued and skipped phases are shown explicitly. Also rename user-facing "work units" / "Failed work units" to "tasks" / "Failed tasks" in ingest output and parseIngestSummary. The parser still accepts the legacy "Work units:" wording in captured output for backward compat. Internal memory-flow event names and type fields are left alone. * Fix test harness failures * Fix CI smoke checks --------- Co-authored-by: Andrey Avtomonov <7889985+andreybavt@users.noreply.github.com>
This commit is contained in:
Andrey Avtomonov
GitHub
parent
1a472cf3ed
commit
b00c1a11a9
118 changed files with 16890 additions and 2992 deletions
|
|
@ -1,57 +1,22 @@
|
|||
import { resolve } from 'node:path';
|
||||
import { type Command, Option } from '@commander-js/extra-typings';
|
||||
import { collectOption, type KtxCliCommandContext, type OutputModeOptions, resolveCommandProjectDir } from '../cli-program.js';
|
||||
import {
|
||||
collectOption,
|
||||
type KtxCliCommandContext,
|
||||
parsePositiveIntegerOption,
|
||||
resolveCommandProjectDir,
|
||||
} from '../cli-program.js';
|
||||
import type { KtxCliDeps, KtxCliIo } from '../index.js';
|
||||
import type { KtxIngestArgs, KtxIngestOutputMode } from '../ingest.js';
|
||||
import { runtimeInstallPolicyFromFlags } from '../managed-python-command.js';
|
||||
import type { KtxPublicIngestArgs } from '../public-ingest.js';
|
||||
import { profileMark } from '../startup-profile.js';
|
||||
import type { KtxTextIngestArgs } from '../text-ingest.js';
|
||||
|
||||
profileMark('module:commands/ingest-commands');
|
||||
|
||||
interface IngestCommandOptions {
|
||||
runIngestWithProgress: (
|
||||
args: KtxIngestArgs,
|
||||
io: KtxCliIo,
|
||||
deps: KtxCliDeps,
|
||||
defaultRunIngest: (args: KtxIngestArgs, io: KtxCliIo) => Promise<number>,
|
||||
) => Promise<number>;
|
||||
runTextIngest: (args: KtxTextIngestArgs, io: KtxCliIo, deps: KtxCliDeps) => Promise<number>;
|
||||
}
|
||||
|
||||
function outputMode(options: OutputModeOptions): KtxIngestOutputMode {
|
||||
if (options.json === true) {
|
||||
return 'json';
|
||||
}
|
||||
if (options.viz === true) {
|
||||
return 'viz';
|
||||
}
|
||||
return 'plain';
|
||||
}
|
||||
|
||||
function watchOutputMode(options: OutputModeOptions): KtxIngestOutputMode {
|
||||
if (options.json === true) {
|
||||
return 'json';
|
||||
}
|
||||
if (options.plain === true) {
|
||||
return 'plain';
|
||||
}
|
||||
return 'viz';
|
||||
}
|
||||
|
||||
function inputMode(options: OutputModeOptions): Pick<KtxIngestArgs, 'inputMode'> {
|
||||
return options.input === false ? { inputMode: 'disabled' } : {};
|
||||
}
|
||||
|
||||
async function runIngestArgs(
|
||||
context: KtxCliCommandContext,
|
||||
args: KtxIngestArgs,
|
||||
options: IngestCommandOptions,
|
||||
): Promise<void> {
|
||||
const { runKtxIngest } = await import('../ingest.js');
|
||||
context.setExitCode(await options.runIngestWithProgress(args, context.io, context.deps, runKtxIngest));
|
||||
}
|
||||
|
||||
export function registerIngestCommands(
|
||||
program: Command,
|
||||
context: KtxCliCommandContext,
|
||||
|
|
@ -59,50 +24,45 @@ export function registerIngestCommands(
|
|||
): void {
|
||||
const ingest = program
|
||||
.command('ingest')
|
||||
.description('Run or inspect local ingest memory-flow output')
|
||||
.description('Build or inspect KTX context')
|
||||
.usage('[options] [connectionId]')
|
||||
.argument('[connectionId]', 'Configured connection id to ingest')
|
||||
.option('--all', 'Ingest all configured connections', false)
|
||||
.addOption(new Option('--fast', 'Use deterministic database schema ingest').conflicts('deep'))
|
||||
.addOption(new Option('--deep', 'Use AI-enriched database ingest').conflicts('fast'))
|
||||
.addOption(new Option('--query-history', 'Include database query-history usage patterns').conflicts('noQueryHistory'))
|
||||
.addOption(new Option('--no-query-history', 'Skip database query-history usage patterns'))
|
||||
.option('--query-history-window-days <days>', 'Query-history lookback window for this run', parsePositiveIntegerOption)
|
||||
.addOption(new Option('--plain', 'Print plain text output').conflicts(['json']))
|
||||
.addOption(new Option('--json', 'Print JSON output').conflicts(['plain']))
|
||||
.option('--no-input', 'Disable interactive terminal input')
|
||||
.showHelpAfterError();
|
||||
|
||||
ingest.action(async (connectionId: string | undefined, options, command) => {
|
||||
const { runKtxPublicIngest } = await import('../public-ingest.js');
|
||||
const queryHistory =
|
||||
options.queryHistory === true ? 'enabled' : options.queryHistory === false ? 'disabled' : 'default';
|
||||
const args: KtxPublicIngestArgs = {
|
||||
command: 'run',
|
||||
projectDir: resolveCommandProjectDir(command),
|
||||
...(connectionId ? { targetConnectionId: connectionId } : {}),
|
||||
all: options.all === true,
|
||||
json: options.json === true,
|
||||
inputMode: options.input === false ? 'disabled' : 'auto',
|
||||
...(options.fast === true ? { depth: 'fast' as const } : {}),
|
||||
...(options.deep === true ? { depth: 'deep' as const } : {}),
|
||||
queryHistory,
|
||||
...(options.queryHistoryWindowDays !== undefined ? { queryHistoryWindowDays: options.queryHistoryWindowDays } : {}),
|
||||
cliVersion: context.packageInfo.version,
|
||||
runtimeInstallPolicy: runtimeInstallPolicyFromFlags(options),
|
||||
};
|
||||
context.setExitCode(await (context.deps.publicIngest ?? runKtxPublicIngest)(args, context.io));
|
||||
});
|
||||
|
||||
ingest.hook('preAction', (_thisCommand, actionCommand) => {
|
||||
context.writeDebug?.('ingest', actionCommand);
|
||||
});
|
||||
|
||||
ingest
|
||||
.command('run')
|
||||
.description('Run local ingest for one configured connection and source adapter')
|
||||
.requiredOption('--connection-id <connectionId>', 'KTX connection id')
|
||||
.requiredOption('--adapter <adapter>', 'Ingest source adapter name')
|
||||
.option('--source-dir <path>', 'Directory containing source files')
|
||||
.option('--database-introspection-url <url>', 'Daemon URL for live-database introspection')
|
||||
.option('--debug-llm-request-file <path>', 'Write sanitized LLM request structure to a JSONL file')
|
||||
.option('--report-file <path>', 'Unsupported for ingest run; use ingest status/watch instead')
|
||||
.addOption(new Option('--plain', 'Print plain text output').conflicts(['json', 'viz']))
|
||||
.addOption(new Option('--json', 'Print JSON output').conflicts(['plain', 'viz']))
|
||||
.addOption(new Option('--viz', 'Render memory-flow TUI output').conflicts(['plain', 'json']))
|
||||
.option('--yes', 'Install the managed Python runtime without prompting when required', false)
|
||||
.option('--no-input', 'Disable interactive terminal input for visualization')
|
||||
.action(async (options, command) => {
|
||||
if (options.reportFile) {
|
||||
throw new Error('--report-file is only supported for ingest status/watch');
|
||||
}
|
||||
await runIngestArgs(
|
||||
context,
|
||||
{
|
||||
command: 'run',
|
||||
projectDir: resolveCommandProjectDir(command),
|
||||
connectionId: options.connectionId,
|
||||
adapter: options.adapter,
|
||||
sourceDir: options.sourceDir ? resolve(options.sourceDir) : undefined,
|
||||
databaseIntrospectionUrl: options.databaseIntrospectionUrl || undefined,
|
||||
cliVersion: context.packageInfo.version,
|
||||
runtimeInstallPolicy: runtimeInstallPolicyFromFlags({ yes: options.yes }),
|
||||
...(options.debugLlmRequestFile ? { debugLlmRequestFile: resolve(options.debugLlmRequestFile) } : {}),
|
||||
outputMode: outputMode(options),
|
||||
...inputMode(options),
|
||||
},
|
||||
commandOptions,
|
||||
);
|
||||
});
|
||||
|
||||
ingest
|
||||
.command('text')
|
||||
.description('Ingest free-form text artifacts into KTX memory')
|
||||
|
|
@ -113,6 +73,7 @@ export function registerIngestCommands(
|
|||
.option('--json', 'Print JSON output')
|
||||
.option('--fail-fast', 'Stop after the first failed text item', false)
|
||||
.action(async (files: string[], options, command) => {
|
||||
const parentOptions = command.parent?.opts() as { json?: boolean } | undefined;
|
||||
context.setExitCode(
|
||||
await commandOptions.runTextIngest(
|
||||
{
|
||||
|
|
@ -121,7 +82,7 @@ export function registerIngestCommands(
|
|||
files,
|
||||
...(options.connectionId ? { connectionId: options.connectionId } : {}),
|
||||
userId: options.userId,
|
||||
json: options.json === true,
|
||||
json: options.json === true || parentOptions?.json === true,
|
||||
failFast: options.failFast === true,
|
||||
},
|
||||
context.io,
|
||||
|
|
@ -129,76 +90,4 @@ export function registerIngestCommands(
|
|||
),
|
||||
);
|
||||
});
|
||||
|
||||
ingest
|
||||
.command('status')
|
||||
.description('Print status for the latest or selected stored local ingest run or report file')
|
||||
.argument('[runId]', 'Local ingest run id, report id, run id, or job id')
|
||||
.option('--report-file <path>', 'Bundle ingest report JSON file to render')
|
||||
.addOption(new Option('--plain', 'Print plain text output').conflicts(['json', 'viz']))
|
||||
.addOption(new Option('--json', 'Print JSON output').conflicts(['plain', 'viz']))
|
||||
.addOption(new Option('--viz', 'Render memory-flow TUI output').conflicts(['plain', 'json']))
|
||||
.option('--no-input', 'Disable interactive terminal input for visualization')
|
||||
.action(async (runId: string | undefined, options, command) => {
|
||||
await runIngestArgs(
|
||||
context,
|
||||
{
|
||||
command: 'status',
|
||||
projectDir: resolveCommandProjectDir(command),
|
||||
...(runId ? { runId } : {}),
|
||||
...(options.reportFile ? { reportFile: resolve(options.reportFile) } : {}),
|
||||
outputMode: outputMode(options),
|
||||
...inputMode(options),
|
||||
},
|
||||
commandOptions,
|
||||
);
|
||||
});
|
||||
|
||||
ingest
|
||||
.command('watch')
|
||||
.description('Open the latest or selected stored ingest visual report')
|
||||
.argument('[runId]', 'Local ingest run id, report id, run id, or job id')
|
||||
.option('--report-file <path>', 'Bundle ingest report JSON file to render')
|
||||
.addOption(new Option('--plain', 'Print plain text output').conflicts(['json', 'viz']))
|
||||
.addOption(new Option('--json', 'Print JSON output').conflicts(['plain', 'viz']))
|
||||
.addOption(new Option('--viz', 'Render memory-flow TUI output').conflicts(['plain', 'json']))
|
||||
.option('--no-input', 'Disable interactive terminal input for visualization')
|
||||
.action(async (runId: string | undefined, options, command) => {
|
||||
await runIngestArgs(
|
||||
context,
|
||||
{
|
||||
command: 'watch',
|
||||
projectDir: resolveCommandProjectDir(command),
|
||||
...(runId ? { runId } : {}),
|
||||
...(options.reportFile ? { reportFile: resolve(options.reportFile) } : {}),
|
||||
outputMode: watchOutputMode(options),
|
||||
...inputMode(options),
|
||||
},
|
||||
commandOptions,
|
||||
);
|
||||
});
|
||||
|
||||
ingest
|
||||
.command('replay')
|
||||
.description('Replay a stored ingest run or bundle report through memory-flow output')
|
||||
.argument('<runId>', 'Local ingest run id, report id, run id, or job id')
|
||||
.option('--report-file <path>', 'Bundle ingest report JSON file to render')
|
||||
.addOption(new Option('--plain', 'Print plain text output').conflicts(['json', 'viz']))
|
||||
.addOption(new Option('--json', 'Print JSON output').conflicts(['plain', 'viz']))
|
||||
.addOption(new Option('--viz', 'Render memory-flow TUI output').conflicts(['plain', 'json']))
|
||||
.option('--no-input', 'Disable interactive terminal input for visualization')
|
||||
.action(async (runId: string, options, command) => {
|
||||
await runIngestArgs(
|
||||
context,
|
||||
{
|
||||
command: 'replay',
|
||||
projectDir: resolveCommandProjectDir(command),
|
||||
runId,
|
||||
...(options.reportFile ? { reportFile: resolve(options.reportFile) } : {}),
|
||||
outputMode: outputMode(options),
|
||||
...inputMode(options),
|
||||
},
|
||||
commandOptions,
|
||||
);
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,11 +1,9 @@
|
|||
import { type Command, Option } from '@commander-js/extra-typings';
|
||||
import type { Command } from '@commander-js/extra-typings';
|
||||
import {
|
||||
collectOption,
|
||||
type KtxCliCommandContext,
|
||||
parsePositiveIntegerOption,
|
||||
resolveCommandProjectDir,
|
||||
} from '../cli-program.js';
|
||||
import { wikiWriteCommandSchema } from '../command-schemas.js';
|
||||
import type { KtxKnowledgeArgs } from '../knowledge.js';
|
||||
import { profileMark } from '../startup-profile.js';
|
||||
|
||||
|
|
@ -19,7 +17,7 @@ async function runKnowledgeArgs(context: KtxCliCommandContext, args: KtxKnowledg
|
|||
export function registerWikiCommands(program: Command, context: KtxCliCommandContext): void {
|
||||
const wiki = program
|
||||
.command('wiki')
|
||||
.description('List, read, search, or write local wiki pages')
|
||||
.description('List or search local wiki pages')
|
||||
.showHelpAfterError()
|
||||
.addHelpText(
|
||||
'after',
|
||||
|
|
@ -40,22 +38,6 @@ export function registerWikiCommands(program: Command, context: KtxCliCommandCon
|
|||
});
|
||||
});
|
||||
|
||||
wiki
|
||||
.command('read')
|
||||
.description('Read one local wiki page')
|
||||
.argument('<key>', 'Wiki page key')
|
||||
.option('--json', 'Print JSON output', false)
|
||||
.option('--user-id <id>', 'Local user id', 'local')
|
||||
.action(async (key: string, options: { userId: string; json?: boolean }, command) => {
|
||||
await runKnowledgeArgs(context, {
|
||||
command: 'read',
|
||||
projectDir: resolveCommandProjectDir(command),
|
||||
key,
|
||||
userId: options.userId,
|
||||
json: options.json,
|
||||
});
|
||||
});
|
||||
|
||||
wiki
|
||||
.command('search')
|
||||
.description('Search local wiki pages')
|
||||
|
|
@ -73,31 +55,4 @@ export function registerWikiCommands(program: Command, context: KtxCliCommandCon
|
|||
...(options.limit !== undefined ? { limit: options.limit } : {}),
|
||||
});
|
||||
});
|
||||
|
||||
wiki
|
||||
.command('write')
|
||||
.description('Write one local wiki page')
|
||||
.argument('<key>', 'Wiki page key')
|
||||
.option('--user-id <id>', 'Local user id', 'local')
|
||||
.addOption(new Option('--scope <scope>', 'global or user').choices(['global', 'user']).default('global'))
|
||||
.requiredOption('--summary <summary>', 'Wiki summary')
|
||||
.requiredOption('--content <content>', 'Wiki content')
|
||||
.option('--tag <tag>', 'Wiki tag; repeatable', collectOption, [])
|
||||
.option('--ref <ref>', 'Wiki ref; repeatable', collectOption, [])
|
||||
.option('--sl-ref <ref>', 'Semantic-layer ref; repeatable', collectOption, [])
|
||||
.action(async (key: string, options, command) => {
|
||||
const args = wikiWriteCommandSchema.parse({
|
||||
command: 'write',
|
||||
projectDir: resolveCommandProjectDir(command),
|
||||
key,
|
||||
scope: options.scope === 'user' ? 'USER' : 'GLOBAL',
|
||||
userId: options.userId,
|
||||
summary: options.summary,
|
||||
content: options.content,
|
||||
tags: options.tag,
|
||||
refs: options.ref,
|
||||
slRefs: options.slRef,
|
||||
});
|
||||
await runKnowledgeArgs(context, args);
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,76 +0,0 @@
|
|||
import { type Command, InvalidArgumentError } from '@commander-js/extra-typings';
|
||||
import { type KtxCliCommandContext, resolveCommandProjectDir } from '../cli-program.js';
|
||||
import { runtimeInstallPolicyFromFlags } from '../managed-python-command.js';
|
||||
import type { KtxScanArgs } from '../scan.js';
|
||||
import { profileMark } from '../startup-profile.js';
|
||||
|
||||
profileMark('module:commands/scan-commands');
|
||||
|
||||
async function runScanArgs(context: KtxCliCommandContext, args: KtxScanArgs): Promise<void> {
|
||||
const runner = context.deps.scan ?? (await import('../scan.js')).runKtxScan;
|
||||
context.setExitCode(await runner(args, context.io));
|
||||
}
|
||||
|
||||
type KtxScanModeOption = Extract<KtxScanArgs, { command: 'run' }>['mode'];
|
||||
|
||||
const REMOVED_SCAN_SUBCOMMAND_NAMES = new Set([
|
||||
'status',
|
||||
'report',
|
||||
'relationships',
|
||||
'relationship-apply',
|
||||
'relationship-feedback',
|
||||
'relationship-calibration',
|
||||
'relationship-thresholds',
|
||||
]);
|
||||
|
||||
function parseScanModeOption(value: string): KtxScanModeOption {
|
||||
if (value === 'structural' || value === 'enriched' || value === 'relationships') {
|
||||
return value;
|
||||
}
|
||||
throw new InvalidArgumentError('Allowed choices are structural, enriched, relationships');
|
||||
}
|
||||
|
||||
function parseConnectionId(value: string): string {
|
||||
if (REMOVED_SCAN_SUBCOMMAND_NAMES.has(value)) {
|
||||
throw new InvalidArgumentError(`"${value}" is not a scan connection id`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
export function registerScanCommands(program: Command, context: KtxCliCommandContext): void {
|
||||
program
|
||||
.command('scan')
|
||||
.description('Run a standalone connection scan')
|
||||
.argument('<connectionId>', 'KTX connection id to scan', parseConnectionId)
|
||||
.option(
|
||||
'--mode <mode>',
|
||||
'Scan mode: structural, enriched, relationships (default: structural)',
|
||||
parseScanModeOption,
|
||||
)
|
||||
.option('--dry-run', 'Run without writing scan results', false)
|
||||
.option('--database-introspection-url <url>', 'Daemon URL for live-database introspection')
|
||||
.option('--yes', 'Install the managed Python runtime without prompting when required', false)
|
||||
.option('--no-input', 'Disable interactive managed runtime installation')
|
||||
.showHelpAfterError()
|
||||
.addHelpText(
|
||||
'after',
|
||||
'\nProject directory defaults to KTX_PROJECT_DIR when set, otherwise the current working directory.\n',
|
||||
)
|
||||
.hook('preAction', (_thisCommand, actionCommand) => {
|
||||
context.writeDebug?.('scan', actionCommand);
|
||||
})
|
||||
.action(async (connectionId: string, options, command) => {
|
||||
const mode = options.mode ?? 'structural';
|
||||
await runScanArgs(context, {
|
||||
command: 'run',
|
||||
projectDir: resolveCommandProjectDir(command),
|
||||
connectionId,
|
||||
mode,
|
||||
detectRelationships: mode === 'relationships',
|
||||
dryRun: options.dryRun === true,
|
||||
databaseIntrospectionUrl: options.databaseIntrospectionUrl,
|
||||
cliVersion: context.packageInfo.version,
|
||||
runtimeInstallPolicy: runtimeInstallPolicyFromFlags(options),
|
||||
});
|
||||
});
|
||||
}
|
||||
|
|
@ -109,12 +109,12 @@ function shouldShowSetupEntryMenu(
|
|||
newDatabaseConnectionId?: string;
|
||||
databaseUrl?: string;
|
||||
databaseSchema?: string[];
|
||||
enableHistoricSql?: boolean;
|
||||
disableHistoricSql?: boolean;
|
||||
historicSqlWindowDays?: number;
|
||||
historicSqlMinExecutions?: number;
|
||||
historicSqlServiceAccountPattern?: string[];
|
||||
historicSqlRedactionPattern?: string[];
|
||||
enableQueryHistory?: boolean;
|
||||
disableQueryHistory?: boolean;
|
||||
queryHistoryWindowDays?: number;
|
||||
queryHistoryMinExecutions?: number;
|
||||
queryHistoryServiceAccountPattern?: string[];
|
||||
queryHistoryRedactionPattern?: string[];
|
||||
skipDatabases?: boolean;
|
||||
source?: KtxSetupSourceType;
|
||||
sourceConnectionId?: string;
|
||||
|
|
@ -147,10 +147,10 @@ function shouldShowSetupEntryMenu(
|
|||
if (options.databaseSchema && options.databaseSchema.length > 0) {
|
||||
return false;
|
||||
}
|
||||
if (options.historicSqlServiceAccountPattern && options.historicSqlServiceAccountPattern.length > 0) {
|
||||
if (options.queryHistoryServiceAccountPattern && options.queryHistoryServiceAccountPattern.length > 0) {
|
||||
return false;
|
||||
}
|
||||
if (options.historicSqlRedactionPattern && options.historicSqlRedactionPattern.length > 0) {
|
||||
if (options.queryHistoryRedactionPattern && options.queryHistoryRedactionPattern.length > 0) {
|
||||
return false;
|
||||
}
|
||||
if (options.notionRootPageId && options.notionRootPageId.length > 0) {
|
||||
|
|
@ -179,10 +179,10 @@ function shouldShowSetupEntryMenu(
|
|||
'skipEmbeddings',
|
||||
'newDatabaseConnectionId',
|
||||
'databaseUrl',
|
||||
'enableHistoricSql',
|
||||
'disableHistoricSql',
|
||||
'historicSqlWindowDays',
|
||||
'historicSqlMinExecutions',
|
||||
'enableQueryHistory',
|
||||
'disableQueryHistory',
|
||||
'queryHistoryWindowDays',
|
||||
'queryHistoryMinExecutions',
|
||||
'skipDatabases',
|
||||
'source',
|
||||
'sourceConnectionId',
|
||||
|
|
@ -282,33 +282,37 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
|
|||
.hideHelp(),
|
||||
)
|
||||
.addOption(
|
||||
new Option('--enable-historic-sql', 'Enable Historic SQL when the selected database supports it')
|
||||
new Option('--enable-query-history', 'Enable query history when the selected database supports it')
|
||||
.hideHelp()
|
||||
.default(false),
|
||||
)
|
||||
.addOption(
|
||||
new Option('--disable-historic-sql', 'Disable Historic SQL for the selected database').hideHelp().default(false),
|
||||
new Option('--disable-query-history', 'Disable query history for the selected database').hideHelp().default(false),
|
||||
)
|
||||
.addOption(new Option('--historic-sql-window-days <number>', 'Historic SQL query-history window').argParser(positiveInteger).hideHelp())
|
||||
.addOption(
|
||||
new Option('--historic-sql-min-executions <number>', 'Minimum Historic SQL executions for a template')
|
||||
new Option('--query-history-window-days <number>', 'Query-history lookback window')
|
||||
.argParser(positiveInteger)
|
||||
.hideHelp(),
|
||||
)
|
||||
.addOption(
|
||||
new Option('--historic-sql-service-account-pattern <pattern>', 'Historic SQL service-account regex; repeatable')
|
||||
new Option('--query-history-min-executions <number>', 'Minimum executions for a query-history template')
|
||||
.argParser(positiveInteger)
|
||||
.hideHelp(),
|
||||
)
|
||||
.addOption(
|
||||
new Option('--query-history-service-account-pattern <pattern>', 'Query-history service-account regex; repeatable')
|
||||
.argParser((value, previous: string[]) => [...previous, value])
|
||||
.default([] as string[])
|
||||
.hideHelp(),
|
||||
)
|
||||
.addOption(
|
||||
new Option('--historic-sql-redaction-pattern <pattern>', 'Historic SQL SQL-literal redaction regex; repeatable')
|
||||
new Option('--query-history-redaction-pattern <pattern>', 'Query-history SQL-literal redaction regex; repeatable')
|
||||
.argParser((value, previous: string[]) => [...previous, value])
|
||||
.default([] as string[])
|
||||
.hideHelp(),
|
||||
)
|
||||
.addOption(
|
||||
new Option('--skip-databases', 'Leave database setup incomplete; KTX cannot work until a primary source is added')
|
||||
new Option('--skip-databases', 'Leave database setup incomplete; KTX cannot work until a database is added')
|
||||
.hideHelp()
|
||||
.default(false),
|
||||
)
|
||||
|
|
@ -371,9 +375,9 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
|
|||
context.setExitCode(1);
|
||||
return;
|
||||
}
|
||||
if (options.enableHistoricSql && options.disableHistoricSql) {
|
||||
if (options.enableQueryHistory && options.disableQueryHistory) {
|
||||
context.io.stderr.write(
|
||||
'Choose only one Historic SQL action: --enable-historic-sql or --disable-historic-sql.\n',
|
||||
'Choose only one query-history action: --enable-query-history or --disable-query-history.\n',
|
||||
);
|
||||
context.setExitCode(1);
|
||||
return;
|
||||
|
|
@ -418,17 +422,17 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
|
|||
...(options.newDatabaseConnectionId ? { databaseConnectionId: options.newDatabaseConnectionId } : {}),
|
||||
...(options.databaseUrl ? { databaseUrl: options.databaseUrl } : {}),
|
||||
databaseSchemas: options.databaseSchema,
|
||||
...(options.enableHistoricSql ? { enableHistoricSql: true } : {}),
|
||||
...(options.disableHistoricSql ? { disableHistoricSql: true } : {}),
|
||||
...(options.historicSqlWindowDays !== undefined ? { historicSqlWindowDays: options.historicSqlWindowDays } : {}),
|
||||
...(options.historicSqlMinExecutions !== undefined
|
||||
? { historicSqlMinExecutions: options.historicSqlMinExecutions }
|
||||
...(options.enableQueryHistory ? { enableQueryHistory: true } : {}),
|
||||
...(options.disableQueryHistory ? { disableQueryHistory: true } : {}),
|
||||
...(options.queryHistoryWindowDays !== undefined ? { queryHistoryWindowDays: options.queryHistoryWindowDays } : {}),
|
||||
...(options.queryHistoryMinExecutions !== undefined
|
||||
? { queryHistoryMinExecutions: options.queryHistoryMinExecutions }
|
||||
: {}),
|
||||
...(options.historicSqlServiceAccountPattern.length > 0
|
||||
? { historicSqlServiceAccountPatterns: options.historicSqlServiceAccountPattern }
|
||||
...(options.queryHistoryServiceAccountPattern.length > 0
|
||||
? { queryHistoryServiceAccountPatterns: options.queryHistoryServiceAccountPattern }
|
||||
: {}),
|
||||
...(options.historicSqlRedactionPattern.length > 0
|
||||
? { historicSqlRedactionPatterns: options.historicSqlRedactionPattern }
|
||||
...(options.queryHistoryRedactionPattern.length > 0
|
||||
? { queryHistoryRedactionPatterns: options.queryHistoryRedactionPattern }
|
||||
: {}),
|
||||
skipDatabases: options.skipDatabases === true,
|
||||
...(options.source ? { source: options.source } : {}),
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue