From f28e12da1d47b26fb7ec6a434b4e3f9f15469f80 Mon Sep 17 00:00:00 2001 From: Andrey Avtomonov Date: Wed, 13 May 2026 19:32:12 +0200 Subject: [PATCH] docs: align ingest CLI reference with unified UX --- .../content/docs/cli-reference/ktx-dev.mdx | 2 +- .../content/docs/cli-reference/ktx-ingest.mdx | 79 ++++++++++--------- .../content/docs/cli-reference/ktx-scan.mdx | 44 ----------- .../content/docs/cli-reference/meta.json | 1 - 4 files changed, 43 insertions(+), 83 deletions(-) delete mode 100644 docs-site/content/docs/cli-reference/ktx-scan.mdx diff --git a/docs-site/content/docs/cli-reference/ktx-dev.mdx b/docs-site/content/docs/cli-reference/ktx-dev.mdx index e00a4585..16a36393 100644 --- a/docs-site/content/docs/cli-reference/ktx-dev.mdx +++ b/docs-site/content/docs/cli-reference/ktx-dev.mdx @@ -3,7 +3,7 @@ title: "ktx dev" description: "Low-level project initialization and runtime management." --- -`ktx dev` contains development-only project initialization and managed runtime commands. Scan and ingest commands live at the root as [`ktx scan`](/docs/cli-reference/ktx-scan) and [`ktx ingest`](/docs/cli-reference/ktx-ingest). +`ktx dev` contains development-only project initialization and managed runtime commands. Context building lives at the root as [`ktx ingest`](/docs/cli-reference/ktx-ingest). ## Command signature diff --git a/docs-site/content/docs/cli-reference/ktx-ingest.mdx b/docs-site/content/docs/cli-reference/ktx-ingest.mdx index e1c0e339..4ae2e6f1 100644 --- a/docs-site/content/docs/cli-reference/ktx-ingest.mdx +++ b/docs-site/content/docs/cli-reference/ktx-ingest.mdx @@ -1,63 +1,67 @@ --- title: "ktx ingest" -description: "Run and inspect local ingest memory-flow output." +description: "Build, inspect, and replay KTX context ingest runs." --- -`ktx ingest` runs adapter-level local ingest and renders stored ingest reports. +`ktx ingest` builds or refreshes KTX context from configured connections. +Database connections build schema context. Context-source connections ingest +metadata from tools such as dbt, Looker, Metabase, MetricFlow, LookML, and +Notion. ## Command signature ```bash -ktx ingest [options] +ktx ingest [options] [connectionId] ``` -## Subcommands +Use a connection id to build one configured connection. Use `--all` to build +every configured connection. Database connections run before context-source +connections when you use `--all`. -| Subcommand | Description | -|-----------|-------------| -| `run` | Run local ingest for one configured connection and source adapter | -| `status [runId]` | Print status for the latest or selected stored local ingest run or report file | -| `watch [runId]` | Open the latest or selected stored ingest visual report | -| `replay ` | Replay a stored ingest run or bundle report through memory-flow output | - -## `ingest run` +## Build options | Flag | Description | Default | |------|-------------|---------| -| `--connection-id ` | KTX connection id | Required | -| `--adapter ` | Ingest source adapter name | Required | -| `--source-dir ` | Directory containing source files | — | -| `--database-introspection-url ` | Daemon URL for live-database introspection | — | -| `--debug-llm-request-file ` | Write sanitized LLM request structure to a JSONL file | — | +| `--all` | Build every configured connection | `false` | +| `--fast` | Use deterministic database schema ingest | Stored connection default, or `fast` | +| `--deep` | Use AI-enriched database ingest | Stored connection default, or `fast` | +| `--query-history` | Include database query-history usage patterns | Stored connection default | +| `--no-query-history` | Skip database query-history usage patterns for this run | Stored connection default | +| `--query-history-window-days ` | Query-history lookback window for this run | Stored connection default | | `--plain` | Print plain text output | `true` | | `--json` | Print JSON output | `false` | -| `--viz` | Render memory-flow TUI output | `false` | -| `--yes` | Install the managed Python runtime without prompting when required | `false` | -| `--no-input` | Disable interactive terminal input for visualization and runtime installation | — | +| `--no-input` | Disable interactive terminal input | `false` | -## `ingest status`, `watch`, and `replay` +`--fast` and `--deep` are mutually exclusive. Depth flags apply only to +database connections. Query-history flags apply only to database connections +that support query history. -| Flag | Description | Default | -|------|-------------|---------| -| `--report-file ` | Bundle ingest report JSON file to render | — | -| `--plain` | Print plain text output | `true` for `status` and `replay` | -| `--json` | Print JSON output | `false` | -| `--viz` | Render memory-flow TUI output | `true` for `watch` | -| `--no-input` | Disable interactive terminal input for visualization | — | +## Status and replay + +| Subcommand | Description | +|------------|-------------| +| `status [runId]` | Print status for the latest or selected stored ingest run or report file | +| `replay ` | Replay a stored ingest run or bundle report through memory-flow output | + +Both subcommands accept `--report-file `, `--plain`, `--json`, `--viz`, +and `--no-input`. ## Examples ```bash -ktx ingest run --connection-id my-dbt-source --adapter dbt -ktx ingest run --connection-id prod-metabase --adapter metabase --yes +ktx ingest warehouse +ktx ingest warehouse --fast +ktx ingest warehouse --deep +ktx ingest warehouse --deep --query-history +ktx ingest warehouse --query-history-window-days 30 +ktx ingest notion +ktx ingest --all +ktx ingest --all --deep ktx ingest status ktx ingest status run-abc123 ktx ingest status --json -ktx ingest watch -ktx ingest watch run-abc123 - ktx ingest replay run-abc123 ktx ingest replay run-abc123 --viz ktx ingest replay run-abc123 --report-file /tmp/ingest-report.json @@ -67,7 +71,8 @@ ktx ingest replay run-abc123 --report-file /tmp/ingest-report.json | Error | Cause | Recovery | |-------|-------|----------| -| Ingest needs credentials | The source adapter requires API or git access | Configure the referenced environment variable or secret file | -| Ingest run cannot find adapter | `--adapter` does not match a supported source adapter | Use a configured adapter such as `dbt`, `metabase`, `looker`, `lookml`, `notion`, or `live-database` | -| Latest run not found | No ingest run has been started in this project | Run `ktx ingest run --connection-id --adapter ` first | -| Report watch fails in a non-interactive shell | Visual report needs a terminal | Use `ktx ingest status --json` for agent and CI workflows | +| Connection not configured | The connection id is not present in `ktx.yaml` | Add the connection with `ktx setup` or update `ktx.yaml` | +| Deep readiness is missing | `--deep` or query history needs model, embedding, and scan-enrichment configuration | Run `ktx setup` or rerun with `--fast` | +| Query history is unsupported | The selected database driver does not support query history | Run schema ingest without query-history flags | +| Latest run not found | No stored ingest report exists in this project | Run `ktx ingest ` first | +| Visual replay fails in a non-interactive shell | Visual report replay needs a terminal | Use `ktx ingest status --json` for agent and CI workflows | diff --git a/docs-site/content/docs/cli-reference/ktx-scan.mdx b/docs-site/content/docs/cli-reference/ktx-scan.mdx deleted file mode 100644 index 2f73ed99..00000000 --- a/docs-site/content/docs/cli-reference/ktx-scan.mdx +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: "ktx scan" -description: "Run standalone database scans." ---- - -Discover a configured database connection's schema, including tables, columns, types, constraints, and optional relationship signals. - -## Command signature - -```bash -ktx scan [options] -``` - -## Options - -| Flag | Description | Default | -|------|-------------|---------| -| `--mode ` | Scan mode: `structural`, `enriched`, or `relationships` | `structural` | -| `--dry-run` | Run without writing scan results | `false` | -| `--database-introspection-url ` | Daemon URL for live-database introspection | — | -| `--yes` | Install the managed Python runtime without prompting when required | `false` | -| `--no-input` | Disable interactive managed runtime installation | — | - -## Examples - -```bash -ktx scan my-warehouse -ktx scan my-warehouse --mode enriched -ktx scan my-warehouse --mode relationships -ktx scan my-warehouse --dry-run -ktx scan my-warehouse --database-introspection-url http://127.0.0.1:8765 -``` - -## Output - -`ktx scan` prints a human summary and writes scan artifacts under the KTX project directory unless `--dry-run` is set. Use `ktx status` after a scan to inspect project readiness and next setup work. - -## Common errors - -| Error | Cause | Recovery | -|-------|-------|----------| -| Scan cannot connect | Connection credentials or network access are invalid | Run `ktx connection test ` and update the connection before scanning | -| Enriched scan cannot describe columns | LLM credentials are missing or invalid | Complete LLM setup with `ktx setup` before enriched scans | -| Relationship scan has limited evidence | The connector cannot provide optional validation or statistics | Re-run with a connector that supports the missing capability, or treat relationship output as lower-confidence context | diff --git a/docs-site/content/docs/cli-reference/meta.json b/docs-site/content/docs/cli-reference/meta.json index bed3f98c..093f75ae 100644 --- a/docs-site/content/docs/cli-reference/meta.json +++ b/docs-site/content/docs/cli-reference/meta.json @@ -4,7 +4,6 @@ "pages": [ "ktx-setup", "ktx-connection", - "ktx-scan", "ktx-ingest", "ktx-sl", "ktx-wiki",