ktx/packages/cli/src/telemetry/command-hook.ts

105 lines
3.4 KiB
TypeScript
Raw Normal View History

import { formatErrorDetail, scrubErrorClass } from './scrubber.js';
feat(telemetry): anonymous posthog usage telemetry across node cli and python daemon (#205) * feat: add telemetry phase 1 * feat: add node telemetry event catalog * feat: add telemetry event helpers * feat: emit setup and connection telemetry * feat: emit connection and stack telemetry * feat: emit ingest and scan telemetry * feat: emit query telemetry * feat: emit sampled mcp telemetry * docs: expand telemetry event catalog * feat: add telemetry schema sync artifact * feat: pass telemetry project id to semantic daemon * feat: add daemon telemetry foundation * feat: emit semantic daemon telemetry * feat: emit daemon lifecycle telemetry * docs: document full telemetry event catalog * feat(telemetry): dim first-run notice * feat(telemetry): show first-run notice before command output * feat(telemetry): wire ktx PostHog project for live ingestion * docs(telemetry): drop posthog project name and host from storage section * docs(telemetry): trim to general overview and disclaimer * docs(agents): add short telemetry guidelines * feat(telemetry): enable posthog geoip enrichment * docs(telemetry): drop ip-geoip note from public overview * refactor(telemetry): drop no-op groupIdentify, rely on capture groups field * fix(telemetry): respect CI kill switch in python daemon identity * fix(sql): route table-count analysis to existing analyze-batch endpoint * fix(telemetry): emit install_first_run from notice path and derive flagsPresent from commander * fix(telemetry): read package info via getKtxCliPackageInfo to satisfy boundary check * fix(telemetry): make python identity env={} bypass os.environ and unset CI in tests * fix(telemetry): unset CI kill switch in cli-program-telemetry tests
2026-05-22 18:18:47 +02:00
export type CommandOutcome = 'ok' | 'error' | 'aborted';
interface CommandSpan {
commandPath: string[];
flagsPresent: Record<string, boolean>;
projectDir?: string;
hasProject: boolean;
attachProjectGroup: boolean;
startedAt: number;
fix(cli): classify ktx setup abandonment as aborted, not a blank error (#278) * fix(cli): classify ktx setup abandonment as aborted, not a blank error ktx setup returned a non-zero exit code without throwing when a user abandoned the interactive wizard, so the command telemetry recorded outcome=error with no errorClass/errorDetail — an unactionable blank in the errors dashboard, where most ktx setup "errors" were really people backing out of the wizard. Add annotateCommandOutcome() to the command span so the setup flow (the decision-maker) records the true outcome: genuine step failures and --no-input missing input become outcome=error with a self-diagnosing reason, while interactive abandonment and project cancellation become outcome=aborted and drop out of the error view. Unify the exit code and telemetry through setupTerminalOutcome() so they can never diverge: aborts now exit 0 (matching the entry-menu Exit, project cancel, and a confirmed Ctrl+C), while failures and automation errors still exit 1. * fix(cli): treat non-TTY setup missing-input as an error, not an abort setupTerminalOutcome classified `missing-input` by `args.inputMode`, but `auto` only means "interactive if a TTY is attached". A piped/CI `ktx setup` without `--no-input` and without `--yes` is still `auto`, yet the project and agents steps return `missing-input` there without ever prompting (e.g. "pass --yes to create a project outside an interactive terminal"). Classifying that as `aborted` made a broken automation run exit 0 — a silent failure. Key the classification off actual interactivity instead: input enabled AND `io.stdout.isTTY === true`. Non-interactive missing-input now exits 1 with a `KtxSetupMissingInput` reason; only a genuine interactive abort exits 0. Adds a non-TTY regression test and fixes the abandonment test to use a real TTY.
2026-06-09 12:53:15 +02:00
annotatedOutcome?: CommandOutcome;
annotatedErrorClass?: string;
annotatedErrorDetail?: string;
feat(telemetry): anonymous posthog usage telemetry across node cli and python daemon (#205) * feat: add telemetry phase 1 * feat: add node telemetry event catalog * feat: add telemetry event helpers * feat: emit setup and connection telemetry * feat: emit connection and stack telemetry * feat: emit ingest and scan telemetry * feat: emit query telemetry * feat: emit sampled mcp telemetry * docs: expand telemetry event catalog * feat: add telemetry schema sync artifact * feat: pass telemetry project id to semantic daemon * feat: add daemon telemetry foundation * feat: emit semantic daemon telemetry * feat: emit daemon lifecycle telemetry * docs: document full telemetry event catalog * feat(telemetry): dim first-run notice * feat(telemetry): show first-run notice before command output * feat(telemetry): wire ktx PostHog project for live ingestion * docs(telemetry): drop posthog project name and host from storage section * docs(telemetry): trim to general overview and disclaimer * docs(agents): add short telemetry guidelines * feat(telemetry): enable posthog geoip enrichment * docs(telemetry): drop ip-geoip note from public overview * refactor(telemetry): drop no-op groupIdentify, rely on capture groups field * fix(telemetry): respect CI kill switch in python daemon identity * fix(sql): route table-count analysis to existing analyze-batch endpoint * fix(telemetry): emit install_first_run from notice path and derive flagsPresent from commander * fix(telemetry): read package info via getKtxCliPackageInfo to satisfy boundary check * fix(telemetry): make python identity env={} bypass os.environ and unset CI in tests * fix(telemetry): unset CI kill switch in cli-program-telemetry tests
2026-05-22 18:18:47 +02:00
}
export interface CompletedCommandSpan {
commandPath: string[];
durationMs: number;
outcome: CommandOutcome;
errorClass?: string;
errorDetail?: string;
feat(telemetry): anonymous posthog usage telemetry across node cli and python daemon (#205) * feat: add telemetry phase 1 * feat: add node telemetry event catalog * feat: add telemetry event helpers * feat: emit setup and connection telemetry * feat: emit connection and stack telemetry * feat: emit ingest and scan telemetry * feat: emit query telemetry * feat: emit sampled mcp telemetry * docs: expand telemetry event catalog * feat: add telemetry schema sync artifact * feat: pass telemetry project id to semantic daemon * feat: add daemon telemetry foundation * feat: emit semantic daemon telemetry * feat: emit daemon lifecycle telemetry * docs: document full telemetry event catalog * feat(telemetry): dim first-run notice * feat(telemetry): show first-run notice before command output * feat(telemetry): wire ktx PostHog project for live ingestion * docs(telemetry): drop posthog project name and host from storage section * docs(telemetry): trim to general overview and disclaimer * docs(agents): add short telemetry guidelines * feat(telemetry): enable posthog geoip enrichment * docs(telemetry): drop ip-geoip note from public overview * refactor(telemetry): drop no-op groupIdentify, rely on capture groups field * fix(telemetry): respect CI kill switch in python daemon identity * fix(sql): route table-count analysis to existing analyze-batch endpoint * fix(telemetry): emit install_first_run from notice path and derive flagsPresent from commander * fix(telemetry): read package info via getKtxCliPackageInfo to satisfy boundary check * fix(telemetry): make python identity env={} bypass os.environ and unset CI in tests * fix(telemetry): unset CI kill switch in cli-program-telemetry tests
2026-05-22 18:18:47 +02:00
flagsPresent: Record<string, boolean>;
hasProject: boolean;
projectDir?: string;
projectGroupAttached: boolean;
}
let activeCommandSpan: CommandSpan | undefined;
export function beginCommandSpan(input: CommandSpan): void {
activeCommandSpan = input;
}
fix(cli): classify ktx setup abandonment as aborted, not a blank error (#278) * fix(cli): classify ktx setup abandonment as aborted, not a blank error ktx setup returned a non-zero exit code without throwing when a user abandoned the interactive wizard, so the command telemetry recorded outcome=error with no errorClass/errorDetail — an unactionable blank in the errors dashboard, where most ktx setup "errors" were really people backing out of the wizard. Add annotateCommandOutcome() to the command span so the setup flow (the decision-maker) records the true outcome: genuine step failures and --no-input missing input become outcome=error with a self-diagnosing reason, while interactive abandonment and project cancellation become outcome=aborted and drop out of the error view. Unify the exit code and telemetry through setupTerminalOutcome() so they can never diverge: aborts now exit 0 (matching the entry-menu Exit, project cancel, and a confirmed Ctrl+C), while failures and automation errors still exit 1. * fix(cli): treat non-TTY setup missing-input as an error, not an abort setupTerminalOutcome classified `missing-input` by `args.inputMode`, but `auto` only means "interactive if a TTY is attached". A piped/CI `ktx setup` without `--no-input` and without `--yes` is still `auto`, yet the project and agents steps return `missing-input` there without ever prompting (e.g. "pass --yes to create a project outside an interactive terminal"). Classifying that as `aborted` made a broken automation run exit 0 — a silent failure. Key the classification off actual interactivity instead: input enabled AND `io.stdout.isTTY === true`. Non-interactive missing-input now exits 1 with a `KtxSetupMissingInput` reason; only a genuine interactive abort exits 0. Adds a non-TTY regression test and fixes the abandonment test to use a real TTY.
2026-06-09 12:53:15 +02:00
/**
* Let a command action record the true outcome and reason on the active span.
*
* The Commander wrapper can only derive an outcome from a thrown error or the
* process exit code, so a command that exits non-zero *without throwing* (e.g.
* `ktx setup` when the user abandons the wizard) lands as `outcome: 'error'`
* with no `errorClass`/`errorDetail` an unactionable blank in the dashboard.
* The action is the decision-maker: it can mark the run `aborted`, or attach a
* scrubbed reason so the next occurrence is self-diagnosing. A later thrown
* error still wins (see {@link completeCommandSpan}), since that is the most
* authoritative signal and also feeds the `$exception` stream. No-ops when no
* span is active so call sites stay safe in tests and bare-help paths.
*
* Values are emitted verbatim and must already satisfy the telemetry privacy
* rules pass synthetic or already-scrubbed strings, never raw user input.
*/
export function annotateCommandOutcome(input: {
outcome?: CommandOutcome;
errorClass?: string;
errorDetail?: string;
}): void {
if (!activeCommandSpan) {
return;
}
if (input.outcome !== undefined) {
activeCommandSpan.annotatedOutcome = input.outcome;
}
if (input.errorClass !== undefined) {
activeCommandSpan.annotatedErrorClass = input.errorClass;
}
if (input.errorDetail !== undefined) {
activeCommandSpan.annotatedErrorDetail = input.errorDetail;
}
}
feat(telemetry): anonymous posthog usage telemetry across node cli and python daemon (#205) * feat: add telemetry phase 1 * feat: add node telemetry event catalog * feat: add telemetry event helpers * feat: emit setup and connection telemetry * feat: emit connection and stack telemetry * feat: emit ingest and scan telemetry * feat: emit query telemetry * feat: emit sampled mcp telemetry * docs: expand telemetry event catalog * feat: add telemetry schema sync artifact * feat: pass telemetry project id to semantic daemon * feat: add daemon telemetry foundation * feat: emit semantic daemon telemetry * feat: emit daemon lifecycle telemetry * docs: document full telemetry event catalog * feat(telemetry): dim first-run notice * feat(telemetry): show first-run notice before command output * feat(telemetry): wire ktx PostHog project for live ingestion * docs(telemetry): drop posthog project name and host from storage section * docs(telemetry): trim to general overview and disclaimer * docs(agents): add short telemetry guidelines * feat(telemetry): enable posthog geoip enrichment * docs(telemetry): drop ip-geoip note from public overview * refactor(telemetry): drop no-op groupIdentify, rely on capture groups field * fix(telemetry): respect CI kill switch in python daemon identity * fix(sql): route table-count analysis to existing analyze-batch endpoint * fix(telemetry): emit install_first_run from notice path and derive flagsPresent from commander * fix(telemetry): read package info via getKtxCliPackageInfo to satisfy boundary check * fix(telemetry): make python identity env={} bypass os.environ and unset CI in tests * fix(telemetry): unset CI kill switch in cli-program-telemetry tests
2026-05-22 18:18:47 +02:00
export function completeCommandSpan(input: {
completedAt: number;
outcome: CommandOutcome;
error?: unknown;
}): CompletedCommandSpan | undefined {
const span = activeCommandSpan;
activeCommandSpan = undefined;
if (!span) {
return undefined;
}
fix(cli): classify ktx setup abandonment as aborted, not a blank error (#278) * fix(cli): classify ktx setup abandonment as aborted, not a blank error ktx setup returned a non-zero exit code without throwing when a user abandoned the interactive wizard, so the command telemetry recorded outcome=error with no errorClass/errorDetail — an unactionable blank in the errors dashboard, where most ktx setup "errors" were really people backing out of the wizard. Add annotateCommandOutcome() to the command span so the setup flow (the decision-maker) records the true outcome: genuine step failures and --no-input missing input become outcome=error with a self-diagnosing reason, while interactive abandonment and project cancellation become outcome=aborted and drop out of the error view. Unify the exit code and telemetry through setupTerminalOutcome() so they can never diverge: aborts now exit 0 (matching the entry-menu Exit, project cancel, and a confirmed Ctrl+C), while failures and automation errors still exit 1. * fix(cli): treat non-TTY setup missing-input as an error, not an abort setupTerminalOutcome classified `missing-input` by `args.inputMode`, but `auto` only means "interactive if a TTY is attached". A piped/CI `ktx setup` without `--no-input` and without `--yes` is still `auto`, yet the project and agents steps return `missing-input` there without ever prompting (e.g. "pass --yes to create a project outside an interactive terminal"). Classifying that as `aborted` made a broken automation run exit 0 — a silent failure. Key the classification off actual interactivity instead: input enabled AND `io.stdout.isTTY === true`. Non-interactive missing-input now exits 1 with a `KtxSetupMissingInput` reason; only a genuine interactive abort exits 0. Adds a non-TTY regression test and fixes the abandonment test to use a real TTY.
2026-06-09 12:53:15 +02:00
// Precedence: a thrown error is authoritative; otherwise an action's own
// annotation; otherwise the wrapper's exit-code-derived outcome.
const thrown = Boolean(input.error);
const outcome = thrown ? input.outcome : (span.annotatedOutcome ?? input.outcome);
const errorClass = thrown ? scrubErrorClass(input.error) : span.annotatedErrorClass;
const errorDetail = thrown ? formatErrorDetail(input.error) : span.annotatedErrorDetail;
feat(telemetry): anonymous posthog usage telemetry across node cli and python daemon (#205) * feat: add telemetry phase 1 * feat: add node telemetry event catalog * feat: add telemetry event helpers * feat: emit setup and connection telemetry * feat: emit connection and stack telemetry * feat: emit ingest and scan telemetry * feat: emit query telemetry * feat: emit sampled mcp telemetry * docs: expand telemetry event catalog * feat: add telemetry schema sync artifact * feat: pass telemetry project id to semantic daemon * feat: add daemon telemetry foundation * feat: emit semantic daemon telemetry * feat: emit daemon lifecycle telemetry * docs: document full telemetry event catalog * feat(telemetry): dim first-run notice * feat(telemetry): show first-run notice before command output * feat(telemetry): wire ktx PostHog project for live ingestion * docs(telemetry): drop posthog project name and host from storage section * docs(telemetry): trim to general overview and disclaimer * docs(agents): add short telemetry guidelines * feat(telemetry): enable posthog geoip enrichment * docs(telemetry): drop ip-geoip note from public overview * refactor(telemetry): drop no-op groupIdentify, rely on capture groups field * fix(telemetry): respect CI kill switch in python daemon identity * fix(sql): route table-count analysis to existing analyze-batch endpoint * fix(telemetry): emit install_first_run from notice path and derive flagsPresent from commander * fix(telemetry): read package info via getKtxCliPackageInfo to satisfy boundary check * fix(telemetry): make python identity env={} bypass os.environ and unset CI in tests * fix(telemetry): unset CI kill switch in cli-program-telemetry tests
2026-05-22 18:18:47 +02:00
return {
commandPath: span.commandPath,
durationMs: Math.max(0, input.completedAt - span.startedAt),
fix(cli): classify ktx setup abandonment as aborted, not a blank error (#278) * fix(cli): classify ktx setup abandonment as aborted, not a blank error ktx setup returned a non-zero exit code without throwing when a user abandoned the interactive wizard, so the command telemetry recorded outcome=error with no errorClass/errorDetail — an unactionable blank in the errors dashboard, where most ktx setup "errors" were really people backing out of the wizard. Add annotateCommandOutcome() to the command span so the setup flow (the decision-maker) records the true outcome: genuine step failures and --no-input missing input become outcome=error with a self-diagnosing reason, while interactive abandonment and project cancellation become outcome=aborted and drop out of the error view. Unify the exit code and telemetry through setupTerminalOutcome() so they can never diverge: aborts now exit 0 (matching the entry-menu Exit, project cancel, and a confirmed Ctrl+C), while failures and automation errors still exit 1. * fix(cli): treat non-TTY setup missing-input as an error, not an abort setupTerminalOutcome classified `missing-input` by `args.inputMode`, but `auto` only means "interactive if a TTY is attached". A piped/CI `ktx setup` without `--no-input` and without `--yes` is still `auto`, yet the project and agents steps return `missing-input` there without ever prompting (e.g. "pass --yes to create a project outside an interactive terminal"). Classifying that as `aborted` made a broken automation run exit 0 — a silent failure. Key the classification off actual interactivity instead: input enabled AND `io.stdout.isTTY === true`. Non-interactive missing-input now exits 1 with a `KtxSetupMissingInput` reason; only a genuine interactive abort exits 0. Adds a non-TTY regression test and fixes the abandonment test to use a real TTY.
2026-06-09 12:53:15 +02:00
outcome,
feat(telemetry): anonymous posthog usage telemetry across node cli and python daemon (#205) * feat: add telemetry phase 1 * feat: add node telemetry event catalog * feat: add telemetry event helpers * feat: emit setup and connection telemetry * feat: emit connection and stack telemetry * feat: emit ingest and scan telemetry * feat: emit query telemetry * feat: emit sampled mcp telemetry * docs: expand telemetry event catalog * feat: add telemetry schema sync artifact * feat: pass telemetry project id to semantic daemon * feat: add daemon telemetry foundation * feat: emit semantic daemon telemetry * feat: emit daemon lifecycle telemetry * docs: document full telemetry event catalog * feat(telemetry): dim first-run notice * feat(telemetry): show first-run notice before command output * feat(telemetry): wire ktx PostHog project for live ingestion * docs(telemetry): drop posthog project name and host from storage section * docs(telemetry): trim to general overview and disclaimer * docs(agents): add short telemetry guidelines * feat(telemetry): enable posthog geoip enrichment * docs(telemetry): drop ip-geoip note from public overview * refactor(telemetry): drop no-op groupIdentify, rely on capture groups field * fix(telemetry): respect CI kill switch in python daemon identity * fix(sql): route table-count analysis to existing analyze-batch endpoint * fix(telemetry): emit install_first_run from notice path and derive flagsPresent from commander * fix(telemetry): read package info via getKtxCliPackageInfo to satisfy boundary check * fix(telemetry): make python identity env={} bypass os.environ and unset CI in tests * fix(telemetry): unset CI kill switch in cli-program-telemetry tests
2026-05-22 18:18:47 +02:00
...(errorClass ? { errorClass } : {}),
...(errorDetail ? { errorDetail } : {}),
feat(telemetry): anonymous posthog usage telemetry across node cli and python daemon (#205) * feat: add telemetry phase 1 * feat: add node telemetry event catalog * feat: add telemetry event helpers * feat: emit setup and connection telemetry * feat: emit connection and stack telemetry * feat: emit ingest and scan telemetry * feat: emit query telemetry * feat: emit sampled mcp telemetry * docs: expand telemetry event catalog * feat: add telemetry schema sync artifact * feat: pass telemetry project id to semantic daemon * feat: add daemon telemetry foundation * feat: emit semantic daemon telemetry * feat: emit daemon lifecycle telemetry * docs: document full telemetry event catalog * feat(telemetry): dim first-run notice * feat(telemetry): show first-run notice before command output * feat(telemetry): wire ktx PostHog project for live ingestion * docs(telemetry): drop posthog project name and host from storage section * docs(telemetry): trim to general overview and disclaimer * docs(agents): add short telemetry guidelines * feat(telemetry): enable posthog geoip enrichment * docs(telemetry): drop ip-geoip note from public overview * refactor(telemetry): drop no-op groupIdentify, rely on capture groups field * fix(telemetry): respect CI kill switch in python daemon identity * fix(sql): route table-count analysis to existing analyze-batch endpoint * fix(telemetry): emit install_first_run from notice path and derive flagsPresent from commander * fix(telemetry): read package info via getKtxCliPackageInfo to satisfy boundary check * fix(telemetry): make python identity env={} bypass os.environ and unset CI in tests * fix(telemetry): unset CI kill switch in cli-program-telemetry tests
2026-05-22 18:18:47 +02:00
flagsPresent: span.flagsPresent,
hasProject: span.hasProject,
projectDir: span.projectDir,
projectGroupAttached: span.attachProjectGroup,
};
}
/** @internal */
export function resetCommandSpan(): void {
activeCommandSpan = undefined;
}