ktx/packages/cli/src/cli-program.ts

513 lines
15 KiB
TypeScript
Raw Normal View History

import { existsSync } from 'node:fs';
import { join } from 'node:path';
2026-05-10 23:12:26 +02:00
import { Command, InvalidArgumentError } from '@commander-js/extra-typings';
2026-05-10 23:51:24 +02:00
import type { KtxCliDeps, KtxCliIo, KtxCliPackageInfo } from './cli-runtime.js';
2026-05-10 23:12:26 +02:00
import { registerConnectionCommands } from './commands/connection-commands.js';
import { registerIngestCommands } from './commands/ingest-commands.js';
2026-05-10 23:12:26 +02:00
import { registerWikiCommands } from './commands/knowledge-commands.js';
import { registerSetupCommands } from './commands/setup-commands.js';
import { registerSlCommands } from './commands/sl-commands.js';
import { registerStatusCommands } from './commands/status-commands.js';
import { registerDevCommands } from './dev.js';
import { renderMissingProjectMessage } from './doctor.js';
2026-05-10 23:51:24 +02:00
import { findNearestKtxProjectDir, resolveKtxProjectDir } from './project-resolver.js';
2026-05-10 23:12:26 +02:00
import { profileMark, profileSpan } from './startup-profile.js';
profileMark('module:cli-program');
2026-05-10 23:51:24 +02:00
export interface KtxCliCommandContext {
io: KtxCliIo;
deps: KtxCliDeps;
feat: npm-managed Python runtime for @kaelio/ktx (#7) * docs: add npm managed python runtime design * build: add bundled python runtime wheel builder * build: make local embedding dependencies optional * build: bundle python runtime wheel in cli artifacts * build: track bundled python runtime release artifact * test: verify bundled python runtime wheel * docs: add plan for bundled python runtime wheel * test: cover managed python runtime lifecycle * feat: add managed python runtime installer * feat: add runtime command runner * feat: expose runtime management commands * test: verify managed python runtime commands * docs: add plan for managed python runtime installer * feat: add managed python command helper * feat: use managed runtime for sl query compute * feat: route sl query managed runtime policy * docs: add plan for managed runtime sl query integration * feat: add managed runtime daemon metadata * feat: manage python daemon lifecycle * feat: add runtime daemon start stop commands * fix: verify managed runtime daemon lifecycle * docs: add plan for managed runtime daemon lifecycle * feat: add managed local embeddings config marker * feat: add managed local embeddings daemon helper * feat: use managed runtime for local embedding setup * feat: pass managed runtime policy through setup * docs: add plan for managed local embeddings runtime * feat: read CLI package metadata dynamically * feat: assemble public kaelio ktx npm package * feat: release one public kaelio ktx npm artifact * test: cover public kaelio ktx package invocations * chore: verify public kaelio ktx package artifacts * docs: add plan for public kaelio ktx npm package * test: verify managed runtime in public package smoke * test: finalize managed runtime release smoke * docs: add plan for managed runtime release smoke * test: specify local embeddings release smoke * feat: add local embeddings runtime smoke * chore: register local embeddings smoke * fix: verify local embeddings smoke * fix: restore artifact smoke python env helper * docs: add plan for managed local embeddings release smoke * refactor: share managed runtime install policy parsing * feat: use managed runtime for agent semantic queries * feat: use managed runtime for MCP semantic compute * docs: add plan for managed agent and MCP semantic runtime * feat(cli): add managed daemon HTTP helpers * feat(cli): route local adapters through managed daemon * feat(cli): use managed daemon for ingest helpers * feat(cli): pass managed daemon options to scan * feat(context): pass MCP ingest pull config options * feat(cli): pass managed daemon options to serve ingest * test: verify managed local ingest daemon runtime * docs: add plan for managed local ingest daemon runtime * docs: align managed runtime examples * docs: add plan for managed runtime docs cleanup * test: cover published package runtime smoke commands * test: validate published package smoke outputs * docs: add plan for published package runtime smoke * build: stamp public npm package version * release: add npm public release policy * release: add guarded npm publish script * release: document public npm release handoff * docs: add plan for public npm release handoff * test: cover managed runtime prune in package smoke * docs: document managed runtime prune * docs: add plan for managed runtime prune smoke and docs * chore: encode uv runtime prerequisite policy * fix: clarify missing uv runtime error * docs: document uv runtime prerequisite * docs: add plan for uv runtime prerequisite contract * refactor: limit release artifacts to public package runtime * chore: align release policy with bundled runtime wheel * docs: describe single public runtime artifact surface * test: verify single public runtime artifact contract * docs: add plan for single public runtime artifact cleanup * fix: align local embeddings smoke with public version * docs: add plan for local embeddings smoke public version * release: soft-launch as @kaelio/ktx@0.1.0-rc.0 on next tag Publish target moves to the pre-release version 0.1.0-rc.0 under the next dist-tag so npm install @kaelio/ktx (which resolves to latest) does not pick up the soft-launch build. Users opt in via @kaelio/ktx@next. * Fix release script boundary checks * Remove PostHog from public package bundle
2026-05-11 15:50:34 +02:00
packageInfo: KtxCliPackageInfo;
2026-05-10 23:12:26 +02:00
setExitCode: (code: number) => void;
2026-05-10 23:51:24 +02:00
runInit: (args: { projectDir: string; projectName?: string; force: boolean }, io: KtxCliIo) => Promise<number>;
2026-05-10 23:12:26 +02:00
writeDebug?: (command: string, commandContext: CommandWithGlobalOptions) => void;
}
export interface OutputModeOptions {
plain?: boolean;
json?: boolean;
viz?: boolean;
input?: boolean;
}
2026-05-10 23:51:24 +02:00
interface KtxCommanderProgramOptions {
runInit: (args: { projectDir: string; projectName?: string; force: boolean }, io: KtxCliIo) => Promise<number>;
2026-05-10 23:12:26 +02:00
}
export interface BuildKtxProgramOptions {
io: KtxCliIo;
deps: KtxCliDeps;
packageInfo: KtxCliPackageInfo;
runInit: (args: { projectDir: string; projectName?: string; force: boolean }, io: KtxCliIo) => Promise<number>;
setExitCode?: (code: number) => void;
}
2026-05-10 23:12:26 +02:00
type CommanderExitLike = { exitCode: number; code: string; message: string };
2026-05-10 23:51:24 +02:00
interface KtxGlobalOptionValues {
2026-05-10 23:12:26 +02:00
projectDir?: string;
debug?: boolean;
}
2026-05-12 15:04:01 +02:00
type CommandPathNode = CommandWithGlobalOptions & {
name: () => string;
parent?: CommandPathNode | null;
};
feat: merge ingest and scan * docs: add CLI component reuse guidance * docs: add unified ingest ux design * Refine unified ingest UX design after adversarial review iteration 1 * Refine unified ingest UX design after adversarial review iteration 2 * Refine unified ingest UX design after adversarial review iteration 3 * feat(cli): route public connection ingest command * feat(cli): hide standalone scan from public help * feat(cli): plan public ingest depth and query history * feat(cli): execute public database ingest facets * feat(ingest): read connection query history config * fix(cli): use public ingest wording * fix(config): stop generating ingest adapter allow lists * docs: document public ingest command * test: align ingest surface expectations * docs: add unified ingest public CLI surface plan * feat(cli): preflight deep public ingest readiness * feat(setup): store query history in connection context * feat(setup): store database context depth * feat(setup): verify context readiness by database depth * fix(setup): keep context build foreground only * fix(config): reject reserved ingest connection ids * test: close unified ingest v1 expectations * docs: add unified ingest v1 closure plan * fix(ingest): bypass adapter allow-list for public source ingest * fix(ingest): honor query history window intent * fix(ingest): hide scan internals from public database ingest * feat(ingest): use foreground view for interactive public ingest * fix(setup): use schema context and query history wording * test(cli): verify unified ingest public output * docs: add unified ingest v1 public output closure plan * fix(setup): forward query history flags * fix(setup): prompt for postgres query history * fix(status): report query history readiness * fix(ingest): remove legacy public guidance * fix(ingest): polish foreground retry copy * docs(examples): use unified query history wording * chore(ingest): finish public query history cleanup * docs: add unified ingest v1 query history status cleanup plan * test(docs): cover unified ingest public docs * docs: align ingest CLI reference with unified UX * docs: update context build guides for unified ingest * docs: update setup and primary source ingest wording * docs: stop advertising adapter-backed example ingest * docs: close unified ingest public docs gaps * docs: add unified ingest v1 docs site closure plan * fix: render unified ingest foreground warnings * fix: explain query history schema order * fix: add public ingest retry guidance * fix: align setup next steps with unified ingest * fix: remove scan wording from demo progress * test: verify unified ingest ux closure * docs: add unified ingest v1 foreground and retry closure plan * fix(cli): preserve query-history pull config in public ingest * fix(cli): omit hidden commands from docs command tree * test(cli): close unified ingest final public surface checks * docs: add unified ingest v1 final public surface closure plan * fix(cli): use public source labels in ingest reports * fix(cli): suppress low-level public ingest output * test(cli): verify unified ingest public plain output * docs: add unified ingest v1 public plain output closure plan * fix(cli): add public ingest copy sanitizers * fix(cli): sanitize public ingest progress copy * fix(cli): rename setup schema scope prompt * docs(plan): add progress copy closure; test: align setup back-nav fixture Adds the iter9 plan and updates the setup back-navigation test fixture to pass disableQueryHistory plus listSchemas/listTables stubs that the unified ingest setup step now requires. * docs(plan): add final ux labels plan with narrowed label scans * fix(cli): aggregate unsupported query-history warnings * fix(cli): align setup database labels * test(cli): fix setup database test type-check * fix(cli): remove primary-source wording from setup output * test(cli): verify unified ingest setup closure * docs(plan): add unified ingest v1 verification copy closure plan * fix(cli): remove top-level scan command * fix(cli): remove legacy ingest and wiki commands * Merge scan into ingest flow * feat(cli): split ingest progress into per-phase rows, rename work units to tasks Each database target in the unified ingest dashboard now renders one row per real subprocess (Schema, then Query history when enabled) instead of a single combined bar. Each phase has its own monotonic 0-100% bar so the progress never snaps back to zero when historic-sql starts after scan completes. Completed phases keep their final bar, summary, and elapsed time visible as an inline audit trail; queued and skipped phases are shown explicitly. Also rename user-facing "work units" / "Failed work units" to "tasks" / "Failed tasks" in ingest output and parseIngestSummary. The parser still accepts the legacy "Work units:" wording in captured output for backward compat. Internal memory-flow event names and type fields are left alone. * Fix test harness failures * Fix CI smoke checks --------- Co-authored-by: Andrey Avtomonov <7889985+andreybavt@users.noreply.github.com>
2026-05-14 01:43:06 +02:00
const PROJECT_AWARE_ROOT_COMMANDS = new Set(['setup', 'connection', 'ingest', 'wiki', 'sl', 'status']);
const COMMANDS_THAT_CREATE_PROJECT = new Set(['setup', 'ktx dev init']);
const COMMANDS_WITH_OWN_MISSING_PROJECT_HANDLING = new Set(['status']);
class KtxProjectMissingAbortError extends Error {
readonly isKtxProjectMissingAbort = true;
constructor() {
super('ktx project missing');
}
}
function isKtxProjectMissingAbortError(error: unknown): error is KtxProjectMissingAbortError {
return (
error instanceof KtxProjectMissingAbortError ||
(typeof error === 'object' && error !== null && (error as { isKtxProjectMissingAbort?: unknown }).isKtxProjectMissingAbort === true)
);
}
feat: merge ingest and scan * docs: add CLI component reuse guidance * docs: add unified ingest ux design * Refine unified ingest UX design after adversarial review iteration 1 * Refine unified ingest UX design after adversarial review iteration 2 * Refine unified ingest UX design after adversarial review iteration 3 * feat(cli): route public connection ingest command * feat(cli): hide standalone scan from public help * feat(cli): plan public ingest depth and query history * feat(cli): execute public database ingest facets * feat(ingest): read connection query history config * fix(cli): use public ingest wording * fix(config): stop generating ingest adapter allow lists * docs: document public ingest command * test: align ingest surface expectations * docs: add unified ingest public CLI surface plan * feat(cli): preflight deep public ingest readiness * feat(setup): store query history in connection context * feat(setup): store database context depth * feat(setup): verify context readiness by database depth * fix(setup): keep context build foreground only * fix(config): reject reserved ingest connection ids * test: close unified ingest v1 expectations * docs: add unified ingest v1 closure plan * fix(ingest): bypass adapter allow-list for public source ingest * fix(ingest): honor query history window intent * fix(ingest): hide scan internals from public database ingest * feat(ingest): use foreground view for interactive public ingest * fix(setup): use schema context and query history wording * test(cli): verify unified ingest public output * docs: add unified ingest v1 public output closure plan * fix(setup): forward query history flags * fix(setup): prompt for postgres query history * fix(status): report query history readiness * fix(ingest): remove legacy public guidance * fix(ingest): polish foreground retry copy * docs(examples): use unified query history wording * chore(ingest): finish public query history cleanup * docs: add unified ingest v1 query history status cleanup plan * test(docs): cover unified ingest public docs * docs: align ingest CLI reference with unified UX * docs: update context build guides for unified ingest * docs: update setup and primary source ingest wording * docs: stop advertising adapter-backed example ingest * docs: close unified ingest public docs gaps * docs: add unified ingest v1 docs site closure plan * fix: render unified ingest foreground warnings * fix: explain query history schema order * fix: add public ingest retry guidance * fix: align setup next steps with unified ingest * fix: remove scan wording from demo progress * test: verify unified ingest ux closure * docs: add unified ingest v1 foreground and retry closure plan * fix(cli): preserve query-history pull config in public ingest * fix(cli): omit hidden commands from docs command tree * test(cli): close unified ingest final public surface checks * docs: add unified ingest v1 final public surface closure plan * fix(cli): use public source labels in ingest reports * fix(cli): suppress low-level public ingest output * test(cli): verify unified ingest public plain output * docs: add unified ingest v1 public plain output closure plan * fix(cli): add public ingest copy sanitizers * fix(cli): sanitize public ingest progress copy * fix(cli): rename setup schema scope prompt * docs(plan): add progress copy closure; test: align setup back-nav fixture Adds the iter9 plan and updates the setup back-navigation test fixture to pass disableQueryHistory plus listSchemas/listTables stubs that the unified ingest setup step now requires. * docs(plan): add final ux labels plan with narrowed label scans * fix(cli): aggregate unsupported query-history warnings * fix(cli): align setup database labels * test(cli): fix setup database test type-check * fix(cli): remove primary-source wording from setup output * test(cli): verify unified ingest setup closure * docs(plan): add unified ingest v1 verification copy closure plan * fix(cli): remove top-level scan command * fix(cli): remove legacy ingest and wiki commands * Merge scan into ingest flow * feat(cli): split ingest progress into per-phase rows, rename work units to tasks Each database target in the unified ingest dashboard now renders one row per real subprocess (Schema, then Query history when enabled) instead of a single combined bar. Each phase has its own monotonic 0-100% bar so the progress never snaps back to zero when historic-sql starts after scan completes. Completed phases keep their final bar, summary, and elapsed time visible as an inline audit trail; queued and skipped phases are shown explicitly. Also rename user-facing "work units" / "Failed work units" to "tasks" / "Failed tasks" in ingest output and parseIngestSummary. The parser still accepts the legacy "Work units:" wording in captured output for backward compat. Internal memory-flow event names and type fields are left alone. * Fix test harness failures * Fix CI smoke checks --------- Co-authored-by: Andrey Avtomonov <7889985+andreybavt@users.noreply.github.com>
2026-05-14 01:43:06 +02:00
const REMOVED_COMMAND_PATHS = new Set([
'scan',
'wiki read',
'wiki write',
]);
const GLOBAL_OPTIONS_WITH_VALUE = new Set(['--project-dir']);
const OPTIONS_WITH_VALUE = new Set([
'--project-dir',
'--query-history-window-days',
'--user-id',
'--limit',
'--format',
'--connection-id',
'--source-name',
'--query-file',
'--max-rows',
]);
2026-05-12 15:04:01 +02:00
2026-05-10 23:12:26 +02:00
export interface CommandWithGlobalOptions {
opts: () => object;
optsWithGlobals?: () => object;
}
function isCommanderExit(error: unknown): error is CommanderExitLike {
return (
typeof error === 'object' &&
error !== null &&
'exitCode' in error &&
typeof (error as { exitCode: unknown }).exitCode === 'number' &&
'code' in error &&
typeof (error as { code: unknown }).code === 'string'
);
}
export function collectOption(value: string, previous: string[] = []): string[] {
return [...previous, value];
}
export function parsePositiveIntegerOption(value: string): number {
const parsed = Number(value);
if (!Number.isInteger(parsed) || parsed < 1) {
throw new InvalidArgumentError('must be a positive integer');
}
return parsed;
}
export function parseNonNegativeIntegerOption(value: string): number {
const parsed = Number(value);
if (!Number.isInteger(parsed) || parsed < 0) {
throw new InvalidArgumentError('must be a non-negative integer');
}
return parsed;
}
export function parseBooleanStringOption(value: string): boolean {
if (value === 'true') {
return true;
}
if (value === 'false') {
return false;
}
throw new InvalidArgumentError('must be true or false');
}
export function parseSafeConnectionIdOption(value: string): string {
if (!/^[a-zA-Z0-9][a-zA-Z0-9_-]*$/.test(value)) {
throw new InvalidArgumentError(`Unsafe connection id: ${value}`);
}
return value;
}
export function parseNonEmptyAssignmentOption(value: string): { key: string; value: string } {
const separatorIndex = value.indexOf('=');
if (separatorIndex <= 0 || separatorIndex === value.length - 1) {
throw new InvalidArgumentError('must be a non-empty <key>=<value> assignment');
}
return {
key: value.slice(0, separatorIndex),
value: value.slice(separatorIndex + 1),
};
}
2026-05-10 23:51:24 +02:00
function optionsWithGlobals(command: CommandWithGlobalOptions): KtxGlobalOptionValues {
2026-05-10 23:12:26 +02:00
const options = command.optsWithGlobals ? command.optsWithGlobals() : command.opts();
const values = options as { projectDir?: unknown; debug?: unknown };
return {
projectDir: typeof values.projectDir === 'string' ? values.projectDir : undefined,
debug: typeof values.debug === 'boolean' ? values.debug : undefined,
};
}
2026-05-12 15:04:01 +02:00
function commandOptions(command: CommandWithGlobalOptions): Record<string, unknown> {
return (command.optsWithGlobals ? command.optsWithGlobals() : command.opts()) as Record<string, unknown>;
}
function commandPath(command: CommandPathNode): string[] {
const path: string[] = [];
let current: CommandPathNode | null | undefined = command;
while (current) {
path.unshift(current.name());
current = current.parent;
}
return path;
}
function isProjectAwareCommand(path: string[]): boolean {
if (path.includes('__complete')) {
return false;
}
const rootCommand = path[1];
if (rootCommand === 'dev') {
return path[2] !== undefined && path[2] !== 'runtime';
2026-05-12 15:04:01 +02:00
}
return rootCommand !== undefined && PROJECT_AWARE_ROOT_COMMANDS.has(rootCommand);
}
function shouldSuppressProjectDirLine(path: string[], options: Record<string, unknown>): boolean {
2026-05-12 23:51:46 +02:00
const commandPathKey = path.join(' ');
if (commandPathKey === 'ktx dev init') {
return true;
}
2026-05-13 09:16:35 -07:00
if (commandPathKey === 'ktx setup') {
return true;
}
2026-05-12 23:51:46 +02:00
if (
commandPathKey === 'ktx status' &&
typeof options.projectDir !== 'string' &&
process.env.KTX_PROJECT_DIR === undefined &&
!findNearestKtxProjectDir(process.cwd())
) {
2026-05-12 15:04:01 +02:00
return true;
}
if (options.viz === true) {
return true;
}
const demoIndex = path.indexOf('demo');
if (demoIndex >= 0) {
const demoCommand = path[demoIndex + 1];
return (
options.json !== true &&
options.plain !== true &&
(demoCommand === undefined || demoCommand === 'replay' || demoCommand === 'ingest')
);
}
return false;
}
function shouldPrintProjectDir(command: CommandPathNode): boolean {
const path = commandPath(command);
if (!isProjectAwareCommand(path)) {
return false;
}
const options = commandOptions(command);
if (options.json === true || options.output === 'json' || options.format === 'json') {
return false;
}
return !shouldSuppressProjectDirLine(path, options);
}
2026-05-10 23:12:26 +02:00
export function resolveCommandProjectDir(command: CommandWithGlobalOptions): string {
2026-05-10 23:51:24 +02:00
return resolveKtxProjectDir({ explicitProjectDir: optionsWithGlobals(command).projectDir });
2026-05-10 23:12:26 +02:00
}
export function resolveCommandProjectDirOverride(command: CommandWithGlobalOptions): string | undefined {
2026-05-10 23:51:24 +02:00
return optionsWithGlobals(command).projectDir ?? process.env.KTX_PROJECT_DIR;
2026-05-10 23:12:26 +02:00
}
2026-05-10 23:51:24 +02:00
function createBaseProgram(info: KtxCliPackageInfo, io: KtxCliIo): Command {
2026-05-10 23:12:26 +02:00
return new Command()
2026-05-10 23:51:24 +02:00
.name('ktx')
.description('KTX data agent context layer CLI')
2026-05-10 23:51:24 +02:00
.option('--project-dir <path>', 'KTX project directory (default: KTX_PROJECT_DIR, nearest ktx.yaml, or cwd)')
2026-05-10 23:12:26 +02:00
.option('--debug', 'Enable diagnostic logging to stderr')
.version(`${info.name} ${info.version}`, '-v, --version', 'Show CLI version')
.helpOption('-h, --help', 'Show this help text')
.configureHelp({ showGlobalOptions: true })
.showHelpAfterError()
.exitOverride()
.configureOutput({
writeOut: (chunk) => io.stdout.write(chunk),
writeErr: (chunk) => io.stderr.write(chunk),
outputError: (chunk, write) => write(chunk),
});
}
2026-05-10 23:51:24 +02:00
function writeDebug(io: KtxCliIo, commandContext: CommandWithGlobalOptions, command: string): void {
2026-05-10 23:12:26 +02:00
const global = optionsWithGlobals(commandContext);
if (global.debug !== true) {
return;
}
io.stderr.write(`[debug] projectDir=${resolveCommandProjectDir(commandContext)}\n`);
io.stderr.write(`[debug] dispatch=${command}\n`);
}
function ktxYamlExists(projectDir: string): boolean {
return existsSync(join(projectDir, 'ktx.yaml'));
}
function commandRendersMissingProjectMessage(path: string[]): boolean {
if (!isProjectAwareCommand(path)) {
return false;
}
const pathKey = path.join(' ');
const rootCommand = path[1];
if (rootCommand !== undefined && COMMANDS_THAT_CREATE_PROJECT.has(rootCommand)) {
return false;
}
if (COMMANDS_THAT_CREATE_PROJECT.has(pathKey)) {
return false;
}
return true;
}
function requiresExistingProject(path: string[]): boolean {
if (!commandRendersMissingProjectMessage(path)) {
return false;
}
const rootCommand = path[1];
if (rootCommand !== undefined && COMMANDS_WITH_OWN_MISSING_PROJECT_HANDLING.has(rootCommand)) {
return false;
}
return true;
}
2026-05-12 15:04:01 +02:00
function writeProjectDir(io: KtxCliIo, commandContext: CommandPathNode): void {
if (!shouldPrintProjectDir(commandContext)) {
return;
}
const projectDir = resolveCommandProjectDir(commandContext);
if (commandRendersMissingProjectMessage(commandPath(commandContext)) && !ktxYamlExists(projectDir)) {
return;
}
io.stderr.write(`Project: ${projectDir}\n`);
}
function ensureProjectAvailable(io: KtxCliIo, command: CommandPathNode): void {
const path = commandPath(command);
if (!requiresExistingProject(path)) {
return;
}
const projectDir = resolveCommandProjectDir(command);
if (ktxYamlExists(projectDir)) {
return;
}
const options = commandOptions(command);
const outputMode: 'plain' | 'json' = options.json === true ? 'json' : 'plain';
renderMissingProjectMessage(projectDir, outputMode, io);
throw new KtxProjectMissingAbortError();
2026-05-12 15:04:01 +02:00
}
2026-05-10 23:12:26 +02:00
function formatCliError(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
feat: merge ingest and scan * docs: add CLI component reuse guidance * docs: add unified ingest ux design * Refine unified ingest UX design after adversarial review iteration 1 * Refine unified ingest UX design after adversarial review iteration 2 * Refine unified ingest UX design after adversarial review iteration 3 * feat(cli): route public connection ingest command * feat(cli): hide standalone scan from public help * feat(cli): plan public ingest depth and query history * feat(cli): execute public database ingest facets * feat(ingest): read connection query history config * fix(cli): use public ingest wording * fix(config): stop generating ingest adapter allow lists * docs: document public ingest command * test: align ingest surface expectations * docs: add unified ingest public CLI surface plan * feat(cli): preflight deep public ingest readiness * feat(setup): store query history in connection context * feat(setup): store database context depth * feat(setup): verify context readiness by database depth * fix(setup): keep context build foreground only * fix(config): reject reserved ingest connection ids * test: close unified ingest v1 expectations * docs: add unified ingest v1 closure plan * fix(ingest): bypass adapter allow-list for public source ingest * fix(ingest): honor query history window intent * fix(ingest): hide scan internals from public database ingest * feat(ingest): use foreground view for interactive public ingest * fix(setup): use schema context and query history wording * test(cli): verify unified ingest public output * docs: add unified ingest v1 public output closure plan * fix(setup): forward query history flags * fix(setup): prompt for postgres query history * fix(status): report query history readiness * fix(ingest): remove legacy public guidance * fix(ingest): polish foreground retry copy * docs(examples): use unified query history wording * chore(ingest): finish public query history cleanup * docs: add unified ingest v1 query history status cleanup plan * test(docs): cover unified ingest public docs * docs: align ingest CLI reference with unified UX * docs: update context build guides for unified ingest * docs: update setup and primary source ingest wording * docs: stop advertising adapter-backed example ingest * docs: close unified ingest public docs gaps * docs: add unified ingest v1 docs site closure plan * fix: render unified ingest foreground warnings * fix: explain query history schema order * fix: add public ingest retry guidance * fix: align setup next steps with unified ingest * fix: remove scan wording from demo progress * test: verify unified ingest ux closure * docs: add unified ingest v1 foreground and retry closure plan * fix(cli): preserve query-history pull config in public ingest * fix(cli): omit hidden commands from docs command tree * test(cli): close unified ingest final public surface checks * docs: add unified ingest v1 final public surface closure plan * fix(cli): use public source labels in ingest reports * fix(cli): suppress low-level public ingest output * test(cli): verify unified ingest public plain output * docs: add unified ingest v1 public plain output closure plan * fix(cli): add public ingest copy sanitizers * fix(cli): sanitize public ingest progress copy * fix(cli): rename setup schema scope prompt * docs(plan): add progress copy closure; test: align setup back-nav fixture Adds the iter9 plan and updates the setup back-navigation test fixture to pass disableQueryHistory plus listSchemas/listTables stubs that the unified ingest setup step now requires. * docs(plan): add final ux labels plan with narrowed label scans * fix(cli): aggregate unsupported query-history warnings * fix(cli): align setup database labels * test(cli): fix setup database test type-check * fix(cli): remove primary-source wording from setup output * test(cli): verify unified ingest setup closure * docs(plan): add unified ingest v1 verification copy closure plan * fix(cli): remove top-level scan command * fix(cli): remove legacy ingest and wiki commands * Merge scan into ingest flow * feat(cli): split ingest progress into per-phase rows, rename work units to tasks Each database target in the unified ingest dashboard now renders one row per real subprocess (Schema, then Query history when enabled) instead of a single combined bar. Each phase has its own monotonic 0-100% bar so the progress never snaps back to zero when historic-sql starts after scan completes. Completed phases keep their final bar, summary, and elapsed time visible as an inline audit trail; queued and skipped phases are shown explicitly. Also rename user-facing "work units" / "Failed work units" to "tasks" / "Failed tasks" in ingest output and parseIngestSummary. The parser still accepts the legacy "Work units:" wording in captured output for backward compat. Internal memory-flow event names and type fields are left alone. * Fix test harness failures * Fix CI smoke checks --------- Co-authored-by: Andrey Avtomonov <7889985+andreybavt@users.noreply.github.com>
2026-05-14 01:43:06 +02:00
function commandPathFromArgv(argv: string[]): string[] {
const path: string[] = [];
for (let index = 0; index < argv.length && path.length < 2; index += 1) {
const arg = argv[index];
if (arg === undefined) {
continue;
}
if (arg === '--') {
break;
}
if ((path.length === 0 ? GLOBAL_OPTIONS_WITH_VALUE : OPTIONS_WITH_VALUE).has(arg)) {
index += 1;
continue;
}
const optionsWithValue = path.length === 0 ? GLOBAL_OPTIONS_WITH_VALUE : OPTIONS_WITH_VALUE;
if ([...optionsWithValue].some((option) => arg.startsWith(`${option}=`))) {
continue;
}
if (path.length === 0 && arg === '--debug') {
continue;
}
if (arg.startsWith('-')) {
continue;
}
path.push(arg);
}
return path;
}
function removedCommandName(argv: string[]): string | null {
const path = commandPathFromArgv(argv);
if (path.length === 0) {
return null;
}
const pathKey = path.join(' ');
return REMOVED_COMMAND_PATHS.has(pathKey) ? path.at(-1) ?? null : null;
}
2026-05-10 23:12:26 +02:00
async function runBareInteractiveCommand(
program: Command,
2026-05-10 23:51:24 +02:00
io: KtxCliIo,
context: KtxCliCommandContext,
2026-05-10 23:12:26 +02:00
): Promise<number> {
2026-05-10 23:51:24 +02:00
const nearestProjectDir = findNearestKtxProjectDir(process.cwd());
const envProjectDir = process.env.KTX_PROJECT_DIR;
const runner = context.deps.setup ?? (await import('./setup.js')).runKtxSetup;
2026-05-10 23:12:26 +02:00
if (!nearestProjectDir && !envProjectDir) {
return await runner(
{
command: 'run',
2026-05-10 23:51:24 +02:00
projectDir: resolveKtxProjectDir(),
2026-05-10 23:12:26 +02:00
mode: 'auto',
agents: false,
agentScope: 'project',
skipAgents: false,
inputMode: 'auto',
yes: false,
feat: npm-managed Python runtime for @kaelio/ktx (#7) * docs: add npm managed python runtime design * build: add bundled python runtime wheel builder * build: make local embedding dependencies optional * build: bundle python runtime wheel in cli artifacts * build: track bundled python runtime release artifact * test: verify bundled python runtime wheel * docs: add plan for bundled python runtime wheel * test: cover managed python runtime lifecycle * feat: add managed python runtime installer * feat: add runtime command runner * feat: expose runtime management commands * test: verify managed python runtime commands * docs: add plan for managed python runtime installer * feat: add managed python command helper * feat: use managed runtime for sl query compute * feat: route sl query managed runtime policy * docs: add plan for managed runtime sl query integration * feat: add managed runtime daemon metadata * feat: manage python daemon lifecycle * feat: add runtime daemon start stop commands * fix: verify managed runtime daemon lifecycle * docs: add plan for managed runtime daemon lifecycle * feat: add managed local embeddings config marker * feat: add managed local embeddings daemon helper * feat: use managed runtime for local embedding setup * feat: pass managed runtime policy through setup * docs: add plan for managed local embeddings runtime * feat: read CLI package metadata dynamically * feat: assemble public kaelio ktx npm package * feat: release one public kaelio ktx npm artifact * test: cover public kaelio ktx package invocations * chore: verify public kaelio ktx package artifacts * docs: add plan for public kaelio ktx npm package * test: verify managed runtime in public package smoke * test: finalize managed runtime release smoke * docs: add plan for managed runtime release smoke * test: specify local embeddings release smoke * feat: add local embeddings runtime smoke * chore: register local embeddings smoke * fix: verify local embeddings smoke * fix: restore artifact smoke python env helper * docs: add plan for managed local embeddings release smoke * refactor: share managed runtime install policy parsing * feat: use managed runtime for agent semantic queries * feat: use managed runtime for MCP semantic compute * docs: add plan for managed agent and MCP semantic runtime * feat(cli): add managed daemon HTTP helpers * feat(cli): route local adapters through managed daemon * feat(cli): use managed daemon for ingest helpers * feat(cli): pass managed daemon options to scan * feat(context): pass MCP ingest pull config options * feat(cli): pass managed daemon options to serve ingest * test: verify managed local ingest daemon runtime * docs: add plan for managed local ingest daemon runtime * docs: align managed runtime examples * docs: add plan for managed runtime docs cleanup * test: cover published package runtime smoke commands * test: validate published package smoke outputs * docs: add plan for published package runtime smoke * build: stamp public npm package version * release: add npm public release policy * release: add guarded npm publish script * release: document public npm release handoff * docs: add plan for public npm release handoff * test: cover managed runtime prune in package smoke * docs: document managed runtime prune * docs: add plan for managed runtime prune smoke and docs * chore: encode uv runtime prerequisite policy * fix: clarify missing uv runtime error * docs: document uv runtime prerequisite * docs: add plan for uv runtime prerequisite contract * refactor: limit release artifacts to public package runtime * chore: align release policy with bundled runtime wheel * docs: describe single public runtime artifact surface * test: verify single public runtime artifact contract * docs: add plan for single public runtime artifact cleanup * fix: align local embeddings smoke with public version * docs: add plan for local embeddings smoke public version * release: soft-launch as @kaelio/ktx@0.1.0-rc.0 on next tag Publish target moves to the pre-release version 0.1.0-rc.0 under the next dist-tag so npm install @kaelio/ktx (which resolves to latest) does not pick up the soft-launch build. Users opt in via @kaelio/ktx@next. * Fix release script boundary checks * Remove PostHog from public package bundle
2026-05-11 15:50:34 +02:00
cliVersion: context.packageInfo.version,
2026-05-10 23:12:26 +02:00
skipLlm: false,
skipEmbeddings: false,
databaseSchemas: [],
skipDatabases: false,
skipSources: false,
},
io,
);
}
program.outputHelp();
return 0;
}
export function buildKtxProgram(options: BuildKtxProgramOptions): Command {
const program = createBaseProgram(options.packageInfo, options.io);
program.hook('preAction', (_thisCommand, actionCommand) => {
writeProjectDir(options.io, actionCommand as CommandPathNode);
ensureProjectAvailable(options.io, actionCommand as CommandPathNode);
});
const context: KtxCliCommandContext = {
io: options.io,
deps: options.deps,
packageInfo: options.packageInfo,
setExitCode: options.setExitCode ?? (() => {}),
runInit: options.runInit,
writeDebug: (command: string, commandContext: CommandWithGlobalOptions) => {
writeDebug(options.io, commandContext, command);
},
};
registerSetupCommands(program, context);
registerConnectionCommands(program, context);
registerIngestCommands(program, context, {
runTextIngest: async (textIngestArgs, ingestIo, ingestDeps) => {
const { runKtxTextIngest } = await import('./text-ingest.js');
return await (ingestDeps.textIngest ?? runKtxTextIngest)(textIngestArgs, ingestIo);
},
});
registerWikiCommands(program, context);
registerSlCommands(program, context);
registerStatusCommands(program, context);
registerDevCommands(program, context);
return program;
}
2026-05-10 23:51:24 +02:00
export async function runCommanderKtxCli(
2026-05-10 23:12:26 +02:00
argv: string[],
2026-05-10 23:51:24 +02:00
io: KtxCliIo,
deps: KtxCliDeps,
info: KtxCliPackageInfo,
options: KtxCommanderProgramOptions,
2026-05-10 23:12:26 +02:00
): Promise<number> {
profileMark('commander:entry');
let exitCode = 0;
const program = buildKtxProgram({
io,
deps,
packageInfo: info,
runInit: options.runInit,
setExitCode: (code: number) => {
exitCode = code;
},
2026-05-12 15:04:01 +02:00
});
profileMark('commander:program-built');
2026-05-10 23:51:24 +02:00
const context: KtxCliCommandContext = {
2026-05-10 23:12:26 +02:00
io,
deps,
feat: npm-managed Python runtime for @kaelio/ktx (#7) * docs: add npm managed python runtime design * build: add bundled python runtime wheel builder * build: make local embedding dependencies optional * build: bundle python runtime wheel in cli artifacts * build: track bundled python runtime release artifact * test: verify bundled python runtime wheel * docs: add plan for bundled python runtime wheel * test: cover managed python runtime lifecycle * feat: add managed python runtime installer * feat: add runtime command runner * feat: expose runtime management commands * test: verify managed python runtime commands * docs: add plan for managed python runtime installer * feat: add managed python command helper * feat: use managed runtime for sl query compute * feat: route sl query managed runtime policy * docs: add plan for managed runtime sl query integration * feat: add managed runtime daemon metadata * feat: manage python daemon lifecycle * feat: add runtime daemon start stop commands * fix: verify managed runtime daemon lifecycle * docs: add plan for managed runtime daemon lifecycle * feat: add managed local embeddings config marker * feat: add managed local embeddings daemon helper * feat: use managed runtime for local embedding setup * feat: pass managed runtime policy through setup * docs: add plan for managed local embeddings runtime * feat: read CLI package metadata dynamically * feat: assemble public kaelio ktx npm package * feat: release one public kaelio ktx npm artifact * test: cover public kaelio ktx package invocations * chore: verify public kaelio ktx package artifacts * docs: add plan for public kaelio ktx npm package * test: verify managed runtime in public package smoke * test: finalize managed runtime release smoke * docs: add plan for managed runtime release smoke * test: specify local embeddings release smoke * feat: add local embeddings runtime smoke * chore: register local embeddings smoke * fix: verify local embeddings smoke * fix: restore artifact smoke python env helper * docs: add plan for managed local embeddings release smoke * refactor: share managed runtime install policy parsing * feat: use managed runtime for agent semantic queries * feat: use managed runtime for MCP semantic compute * docs: add plan for managed agent and MCP semantic runtime * feat(cli): add managed daemon HTTP helpers * feat(cli): route local adapters through managed daemon * feat(cli): use managed daemon for ingest helpers * feat(cli): pass managed daemon options to scan * feat(context): pass MCP ingest pull config options * feat(cli): pass managed daemon options to serve ingest * test: verify managed local ingest daemon runtime * docs: add plan for managed local ingest daemon runtime * docs: align managed runtime examples * docs: add plan for managed runtime docs cleanup * test: cover published package runtime smoke commands * test: validate published package smoke outputs * docs: add plan for published package runtime smoke * build: stamp public npm package version * release: add npm public release policy * release: add guarded npm publish script * release: document public npm release handoff * docs: add plan for public npm release handoff * test: cover managed runtime prune in package smoke * docs: document managed runtime prune * docs: add plan for managed runtime prune smoke and docs * chore: encode uv runtime prerequisite policy * fix: clarify missing uv runtime error * docs: document uv runtime prerequisite * docs: add plan for uv runtime prerequisite contract * refactor: limit release artifacts to public package runtime * chore: align release policy with bundled runtime wheel * docs: describe single public runtime artifact surface * test: verify single public runtime artifact contract * docs: add plan for single public runtime artifact cleanup * fix: align local embeddings smoke with public version * docs: add plan for local embeddings smoke public version * release: soft-launch as @kaelio/ktx@0.1.0-rc.0 on next tag Publish target moves to the pre-release version 0.1.0-rc.0 under the next dist-tag so npm install @kaelio/ktx (which resolves to latest) does not pick up the soft-launch build. Users opt in via @kaelio/ktx@next. * Fix release script boundary checks * Remove PostHog from public package bundle
2026-05-11 15:50:34 +02:00
packageInfo: info,
2026-05-10 23:12:26 +02:00
setExitCode: (code: number) => {
exitCode = code;
},
runInit: options.runInit,
writeDebug: (command: string, commandContext: CommandWithGlobalOptions) => {
writeDebug(io, commandContext, command);
},
};
if (argv.length === 0) {
if (io.stdout.isTTY === true) {
try {
return await runBareInteractiveCommand(program, io, context);
} catch (error) {
io.stderr.write(`${formatCliError(error)}\n`);
return 1;
}
}
program.outputHelp();
return 0;
}
feat: merge ingest and scan * docs: add CLI component reuse guidance * docs: add unified ingest ux design * Refine unified ingest UX design after adversarial review iteration 1 * Refine unified ingest UX design after adversarial review iteration 2 * Refine unified ingest UX design after adversarial review iteration 3 * feat(cli): route public connection ingest command * feat(cli): hide standalone scan from public help * feat(cli): plan public ingest depth and query history * feat(cli): execute public database ingest facets * feat(ingest): read connection query history config * fix(cli): use public ingest wording * fix(config): stop generating ingest adapter allow lists * docs: document public ingest command * test: align ingest surface expectations * docs: add unified ingest public CLI surface plan * feat(cli): preflight deep public ingest readiness * feat(setup): store query history in connection context * feat(setup): store database context depth * feat(setup): verify context readiness by database depth * fix(setup): keep context build foreground only * fix(config): reject reserved ingest connection ids * test: close unified ingest v1 expectations * docs: add unified ingest v1 closure plan * fix(ingest): bypass adapter allow-list for public source ingest * fix(ingest): honor query history window intent * fix(ingest): hide scan internals from public database ingest * feat(ingest): use foreground view for interactive public ingest * fix(setup): use schema context and query history wording * test(cli): verify unified ingest public output * docs: add unified ingest v1 public output closure plan * fix(setup): forward query history flags * fix(setup): prompt for postgres query history * fix(status): report query history readiness * fix(ingest): remove legacy public guidance * fix(ingest): polish foreground retry copy * docs(examples): use unified query history wording * chore(ingest): finish public query history cleanup * docs: add unified ingest v1 query history status cleanup plan * test(docs): cover unified ingest public docs * docs: align ingest CLI reference with unified UX * docs: update context build guides for unified ingest * docs: update setup and primary source ingest wording * docs: stop advertising adapter-backed example ingest * docs: close unified ingest public docs gaps * docs: add unified ingest v1 docs site closure plan * fix: render unified ingest foreground warnings * fix: explain query history schema order * fix: add public ingest retry guidance * fix: align setup next steps with unified ingest * fix: remove scan wording from demo progress * test: verify unified ingest ux closure * docs: add unified ingest v1 foreground and retry closure plan * fix(cli): preserve query-history pull config in public ingest * fix(cli): omit hidden commands from docs command tree * test(cli): close unified ingest final public surface checks * docs: add unified ingest v1 final public surface closure plan * fix(cli): use public source labels in ingest reports * fix(cli): suppress low-level public ingest output * test(cli): verify unified ingest public plain output * docs: add unified ingest v1 public plain output closure plan * fix(cli): add public ingest copy sanitizers * fix(cli): sanitize public ingest progress copy * fix(cli): rename setup schema scope prompt * docs(plan): add progress copy closure; test: align setup back-nav fixture Adds the iter9 plan and updates the setup back-navigation test fixture to pass disableQueryHistory plus listSchemas/listTables stubs that the unified ingest setup step now requires. * docs(plan): add final ux labels plan with narrowed label scans * fix(cli): aggregate unsupported query-history warnings * fix(cli): align setup database labels * test(cli): fix setup database test type-check * fix(cli): remove primary-source wording from setup output * test(cli): verify unified ingest setup closure * docs(plan): add unified ingest v1 verification copy closure plan * fix(cli): remove top-level scan command * fix(cli): remove legacy ingest and wiki commands * Merge scan into ingest flow * feat(cli): split ingest progress into per-phase rows, rename work units to tasks Each database target in the unified ingest dashboard now renders one row per real subprocess (Schema, then Query history when enabled) instead of a single combined bar. Each phase has its own monotonic 0-100% bar so the progress never snaps back to zero when historic-sql starts after scan completes. Completed phases keep their final bar, summary, and elapsed time visible as an inline audit trail; queued and skipped phases are shown explicitly. Also rename user-facing "work units" / "Failed work units" to "tasks" / "Failed tasks" in ingest output and parseIngestSummary. The parser still accepts the legacy "Work units:" wording in captured output for backward compat. Internal memory-flow event names and type fields are left alone. * Fix test harness failures * Fix CI smoke checks --------- Co-authored-by: Andrey Avtomonov <7889985+andreybavt@users.noreply.github.com>
2026-05-14 01:43:06 +02:00
const removedCommand = removedCommandName(argv);
if (removedCommand) {
io.stderr.write(`error: unknown command '${removedCommand}'\n`);
return 1;
}
2026-05-10 23:12:26 +02:00
try {
await profileSpan('commander:parseAsync', () => program.parseAsync(argv, { from: 'user' }));
} catch (error) {
if (isKtxProjectMissingAbortError(error)) {
return 1;
}
2026-05-10 23:12:26 +02:00
if (isCommanderExit(error)) {
return error.exitCode === 0 ? 0 : 1;
}
io.stderr.write(`${formatCliError(error)}\n`);
return 1;
}
return exitCode;
}