2026-05-11 00:45:43 -07:00
---
title: "ktx setup"
2026-05-20 17:33:38 +02:00
description: "Set up or resume a local ktx project."
2026-05-11 00:45:43 -07:00
---
2026-05-20 17:33:38 +02:00
`ktx setup` is the guided configuration flow for a local **ktx** project. It can
2026-05-14 12:53:55 -04:00
create or resume `ktx.yaml`, configure LLM and embedding providers, add
2026-05-17 10:27:29 +02:00
database and context-source connections, prepare required runtime features,
build initial context, and install agent integrations.
2026-05-14 12:53:55 -04:00
2026-05-20 17:33:38 +02:00
When you run bare `ktx` in an interactive terminal outside any **ktx** project, the
2026-05-14 12:53:55 -04:00
CLI starts this same setup flow. Inside an existing project, `ktx setup`
resumes from incomplete setup state or opens the setup menu.
2026-05-11 00:45:43 -07:00
2026-05-11 16:43:08 -07:00
## Command signature
2026-05-11 00:45:43 -07:00
```bash
ktx setup [options]
```
2026-05-14 12:53:55 -04:00
## Visible Options
2026-05-11 00:45:43 -07:00
2026-05-14 12:53:55 -04:00
The help output intentionally keeps setup focused on the common interactive
flags. Automation flags are accepted by the same command and are documented
below.
2026-05-11 00:45:43 -07:00
| Flag | Description | Default |
|------|-------------|---------|
2026-05-19 12:18:52 +02:00
| `--agents` | Install agent configuration and rules only | `false` |
2026-05-19 19:23:35 +02:00
| `--target <target>` | Agent target: `claude-code`, `claude-desktop`, `codex`, `cursor`, `opencode`, or `universal` | - |
2026-05-14 12:53:55 -04:00
| `--global` | Install agent integration into the global target scope for `claude-code` or `codex` | `false` |
2026-05-19 19:23:35 +02:00
| `--yes` | Accept project creation and runtime install defaults where setup asks for confirmation | `false` |
2026-05-14 12:43:14 -04:00
| `--no-input` | Disable interactive terminal input | - |
2026-05-11 00:45:43 -07:00
2026-05-14 12:53:55 -04:00
Use the global `--project-dir <path>` option when setup should target a
specific directory.
## Automation Options
These flags are useful for repeatable setup in examples, tests, CI fixtures, and
scripted project creation. They are not shown in `ktx setup --help`.
2026-05-19 19:23:35 +02:00
### Project Creation
2026-05-11 00:45:43 -07:00
2026-05-19 19:23:35 +02:00
Setup resumes an existing `ktx.yaml` when one is present. When no project
exists, interactive setup prompts for where to create it. In scripts, pass
`--project-dir <dir> --no-input --yes` to create the target directory without
prompts.
2026-05-14 12:53:55 -04:00
### LLM Provider
| Flag | Description |
|------|-------------|
feat: add codex llm backend for ktx runtime work (#253)
* feat: add codex sdk runner foundation
* feat: parse codex runtime events
* feat: expose codex runtime mcp tools
* feat: add codex llm runtime
* feat: wire codex llm backend
* test: avoid Array.fromAsync in codex runner test
* docs: document codex llm backend
* fix: tighten codex runtime config ownership
* fix: use codex sdk env and thread options
* fix: parse codex sdk event shapes
* test: add codex backend live smoke
* docs: clarify codex backend isolation
* fix: drive codex loop metrics from mcp events
* fix: enforce codex local step budget
* docs: disclose codex isolation limits
* fix: count all codex agent steps and stream step callbacks live
The agent-loop step budget only counted completed mcp_tool_call items, so
built-in command_execution steps (which the public Codex SDK/CLI surface can
still expose) never decremented the budget, letting ingest/reconciliation run
past stepBudget until Codex stopped on its own. onStepFinish was also replayed
only after the whole stream drained, so live work_unit_step / reconciliation
progress appeared stuck until the Codex process exited.
collectEvents is now the single live step accumulator: it counts every
completed agent-action item via a shared isCompletedAgentStep predicate
(command_execution, mcp_tool_call, file_change, web_search), fires onStepFinish
as each step completes, and enforces the budget on that broader count. A
no-tool turn still counts as one step. toolFailures stays MCP-specific, since a
non-zero command exit is normal agent exploration, not a loop failure.
* test: align ingest llm-guard assertions with codex backend
The skip-llm ingest guard message now lists codex as a valid backend and
mentions a Claude Code/Codex session plus a codex setup hint, but this slow
suite test still asserted the pre-codex wording. Update it to match the
production message (already covered by the local-bundle-runtime unit test) and
add the codex setup-line assertion.
* fix: treat codex error:null tool calls as success
The Codex SDK serializes error: null on successful mcp_tool_call items, so
the failure check (item.error !== undefined) flagged every successful tool
call as failed with the empty-payload default "Codex turn failed". This
killed every ingest work unit under the codex backend before it could
produce a patch.
Key on status === 'failed' (authoritative, always set) and only treat a
populated error object as a failure. Add a regression test built from a
verbatim real-SDK event capture.
* fix: default codex backend to gpt-5.5 and report real probe errors
The previous default gpt-5.3-codex is an API-key-only model that the OpenAI
API rejects under ChatGPT-account (subscription) auth, so codex status/setup
failed with a misleading "authentication is not usable" message even though
auth was fine.
- Default codex model is now gpt-5.5 (works on both subscription and API-key
auth); the curated setup picker offers gpt-5.5 / gpt-5.4 / gpt-5.4-mini and
keeps free-form entry for account-specific ids (e.g. gpt-5.3-codex-spark).
- runCodexAuthProbe now distinguishes "model not available" from an auth
failure and surfaces the real API error: collectEvents retains stream
events when the SDK throws on a non-zero exit, and the API error JSON
envelope is unwrapped to its human-readable message.
- The Codex isolation warning now renders inside the clack setup frame.
- Docs updated to gpt-5.5 with a note that *-codex ids require API-key auth.
* fix: require llm.models.default in status and match codex probe remediation
Status reported a project ready when a non-none LLM backend was configured
without llm.models.default, but the runtime (resolveModelSlots) hard-requires
it, so ingest/scan/memory threw after `ktx status` said the project was usable.
buildLlmStatus now fails for any non-none backend missing models.default and no
longer invents a fallback model for claude-code/codex.
Codex probe failures now carry a category-matched fix: a model-access failure
steers the user at llm.models.default instead of the auth/install remediation.
runCodexAuthProbe returns the fix and status consumes it; the message stays
self-sufficient so setup output is unchanged.
Docs: README now lists the codex backend and local Codex auth; ktx-setup.mdx
states --llm-model only accepts codex/default or gpt-*/codex-* ids.
Repaired four doctor fixtures that configured a backend without models.default
(the now-correctly-blocked config) and added coverage for the new behavior.
2026-06-02 13:57:11 +02:00
| `--llm-backend <backend>` | LLM backend: `anthropic`, `vertex`, `claude-code`, or `codex` |
2026-05-20 17:33:38 +02:00
| `--llm-backend claude-code` | Use the local Claude Code session for **ktx** LLM calls |
feat: add codex llm backend for ktx runtime work (#253)
* feat: add codex sdk runner foundation
* feat: parse codex runtime events
* feat: expose codex runtime mcp tools
* feat: add codex llm runtime
* feat: wire codex llm backend
* test: avoid Array.fromAsync in codex runner test
* docs: document codex llm backend
* fix: tighten codex runtime config ownership
* fix: use codex sdk env and thread options
* fix: parse codex sdk event shapes
* test: add codex backend live smoke
* docs: clarify codex backend isolation
* fix: drive codex loop metrics from mcp events
* fix: enforce codex local step budget
* docs: disclose codex isolation limits
* fix: count all codex agent steps and stream step callbacks live
The agent-loop step budget only counted completed mcp_tool_call items, so
built-in command_execution steps (which the public Codex SDK/CLI surface can
still expose) never decremented the budget, letting ingest/reconciliation run
past stepBudget until Codex stopped on its own. onStepFinish was also replayed
only after the whole stream drained, so live work_unit_step / reconciliation
progress appeared stuck until the Codex process exited.
collectEvents is now the single live step accumulator: it counts every
completed agent-action item via a shared isCompletedAgentStep predicate
(command_execution, mcp_tool_call, file_change, web_search), fires onStepFinish
as each step completes, and enforces the budget on that broader count. A
no-tool turn still counts as one step. toolFailures stays MCP-specific, since a
non-zero command exit is normal agent exploration, not a loop failure.
* test: align ingest llm-guard assertions with codex backend
The skip-llm ingest guard message now lists codex as a valid backend and
mentions a Claude Code/Codex session plus a codex setup hint, but this slow
suite test still asserted the pre-codex wording. Update it to match the
production message (already covered by the local-bundle-runtime unit test) and
add the codex setup-line assertion.
* fix: treat codex error:null tool calls as success
The Codex SDK serializes error: null on successful mcp_tool_call items, so
the failure check (item.error !== undefined) flagged every successful tool
call as failed with the empty-payload default "Codex turn failed". This
killed every ingest work unit under the codex backend before it could
produce a patch.
Key on status === 'failed' (authoritative, always set) and only treat a
populated error object as a failure. Add a regression test built from a
verbatim real-SDK event capture.
* fix: default codex backend to gpt-5.5 and report real probe errors
The previous default gpt-5.3-codex is an API-key-only model that the OpenAI
API rejects under ChatGPT-account (subscription) auth, so codex status/setup
failed with a misleading "authentication is not usable" message even though
auth was fine.
- Default codex model is now gpt-5.5 (works on both subscription and API-key
auth); the curated setup picker offers gpt-5.5 / gpt-5.4 / gpt-5.4-mini and
keeps free-form entry for account-specific ids (e.g. gpt-5.3-codex-spark).
- runCodexAuthProbe now distinguishes "model not available" from an auth
failure and surfaces the real API error: collectEvents retains stream
events when the SDK throws on a non-zero exit, and the API error JSON
envelope is unwrapped to its human-readable message.
- The Codex isolation warning now renders inside the clack setup frame.
- Docs updated to gpt-5.5 with a note that *-codex ids require API-key auth.
* fix: require llm.models.default in status and match codex probe remediation
Status reported a project ready when a non-none LLM backend was configured
without llm.models.default, but the runtime (resolveModelSlots) hard-requires
it, so ingest/scan/memory threw after `ktx status` said the project was usable.
buildLlmStatus now fails for any non-none backend missing models.default and no
longer invents a fallback model for claude-code/codex.
Codex probe failures now carry a category-matched fix: a model-access failure
steers the user at llm.models.default instead of the auth/install remediation.
runCodexAuthProbe returns the fix and status consumes it; the message stays
self-sufficient so setup output is unchanged.
Docs: README now lists the codex backend and local Codex auth; ktx-setup.mdx
states --llm-model only accepts codex/default or gpt-*/codex-* ids.
Repaired four doctor fixtures that configured a backend without models.default
(the now-correctly-blocked config) and added coverage for the new behavior.
2026-06-02 13:57:11 +02:00
| `--llm-backend codex` | Use local Codex authentication for **ktx** LLM calls |
2026-05-14 12:53:55 -04:00
| `--anthropic-api-key-env <name>` | Environment variable containing the Anthropic API key |
| `--anthropic-api-key-file <path>` | File containing the Anthropic API key |
| `--vertex-project <project>` | Vertex AI project ID, `env:NAME`, or `file:/path` reference |
| `--vertex-location <location>` | Vertex AI location, `env:NAME`, or `file:/path` reference |
| `--skip-llm` | Leave LLM setup incomplete |
Choose only one Anthropic credential source. Anthropic credential flags are only
valid with the Anthropic backend; Vertex flags are only valid with the Vertex
feat: add codex llm backend for ktx runtime work (#253)
* feat: add codex sdk runner foundation
* feat: parse codex runtime events
* feat: expose codex runtime mcp tools
* feat: add codex llm runtime
* feat: wire codex llm backend
* test: avoid Array.fromAsync in codex runner test
* docs: document codex llm backend
* fix: tighten codex runtime config ownership
* fix: use codex sdk env and thread options
* fix: parse codex sdk event shapes
* test: add codex backend live smoke
* docs: clarify codex backend isolation
* fix: drive codex loop metrics from mcp events
* fix: enforce codex local step budget
* docs: disclose codex isolation limits
* fix: count all codex agent steps and stream step callbacks live
The agent-loop step budget only counted completed mcp_tool_call items, so
built-in command_execution steps (which the public Codex SDK/CLI surface can
still expose) never decremented the budget, letting ingest/reconciliation run
past stepBudget until Codex stopped on its own. onStepFinish was also replayed
only after the whole stream drained, so live work_unit_step / reconciliation
progress appeared stuck until the Codex process exited.
collectEvents is now the single live step accumulator: it counts every
completed agent-action item via a shared isCompletedAgentStep predicate
(command_execution, mcp_tool_call, file_change, web_search), fires onStepFinish
as each step completes, and enforces the budget on that broader count. A
no-tool turn still counts as one step. toolFailures stays MCP-specific, since a
non-zero command exit is normal agent exploration, not a loop failure.
* test: align ingest llm-guard assertions with codex backend
The skip-llm ingest guard message now lists codex as a valid backend and
mentions a Claude Code/Codex session plus a codex setup hint, but this slow
suite test still asserted the pre-codex wording. Update it to match the
production message (already covered by the local-bundle-runtime unit test) and
add the codex setup-line assertion.
* fix: treat codex error:null tool calls as success
The Codex SDK serializes error: null on successful mcp_tool_call items, so
the failure check (item.error !== undefined) flagged every successful tool
call as failed with the empty-payload default "Codex turn failed". This
killed every ingest work unit under the codex backend before it could
produce a patch.
Key on status === 'failed' (authoritative, always set) and only treat a
populated error object as a failure. Add a regression test built from a
verbatim real-SDK event capture.
* fix: default codex backend to gpt-5.5 and report real probe errors
The previous default gpt-5.3-codex is an API-key-only model that the OpenAI
API rejects under ChatGPT-account (subscription) auth, so codex status/setup
failed with a misleading "authentication is not usable" message even though
auth was fine.
- Default codex model is now gpt-5.5 (works on both subscription and API-key
auth); the curated setup picker offers gpt-5.5 / gpt-5.4 / gpt-5.4-mini and
keeps free-form entry for account-specific ids (e.g. gpt-5.3-codex-spark).
- runCodexAuthProbe now distinguishes "model not available" from an auth
failure and surfaces the real API error: collectEvents retains stream
events when the SDK throws on a non-zero exit, and the API error JSON
envelope is unwrapped to its human-readable message.
- The Codex isolation warning now renders inside the clack setup frame.
- Docs updated to gpt-5.5 with a note that *-codex ids require API-key auth.
* fix: require llm.models.default in status and match codex probe remediation
Status reported a project ready when a non-none LLM backend was configured
without llm.models.default, but the runtime (resolveModelSlots) hard-requires
it, so ingest/scan/memory threw after `ktx status` said the project was usable.
buildLlmStatus now fails for any non-none backend missing models.default and no
longer invents a fallback model for claude-code/codex.
Codex probe failures now carry a category-matched fix: a model-access failure
steers the user at llm.models.default instead of the auth/install remediation.
runCodexAuthProbe returns the fix and status consumes it; the message stays
self-sufficient so setup output is unchanged.
Docs: README now lists the codex backend and local Codex auth; ktx-setup.mdx
states --llm-model only accepts codex/default or gpt-*/codex-* ids.
Repaired four doctor fixtures that configured a backend without models.default
(the now-correctly-blocked config) and added coverage for the new behavior.
2026-06-02 13:57:11 +02:00
backend. The `claude-code` and `codex` backends use local authentication instead
2026-06-08 15:30:48 +02:00
of Anthropic API key or Vertex flags. After you choose a backend, `ktx setup`
writes that backend's per-role model preset to `ktx.yaml`. To change a model,
edit the matching `llm.models.<role>` value in `ktx.yaml`.
2026-05-14 12:53:55 -04:00
fix(cli): clear error when ktx setup has no LLM backend under --no-input (#281)
* fix(cli): fail clearly when ktx setup has no LLM backend under --no-input
Non-interactive `ktx setup` silently defaulted the LLM backend to `anthropic`
and then failed with `Missing Anthropic API key: pass --anthropic-api-key-env
or --anthropic-api-key-file` — confusing for users who selected a different
provider (e.g. `--target claude-code`) and never asked for the Anthropic API
backend.
That silent default could never succeed: it was reached only when no backend,
Anthropic key, or Vertex flag was supplied, and in exactly that case the
Anthropic credential resolver always failed (no env fallback in disabled mode).
Unlike embeddings, the LLM has no credential-free default (anthropic needs a
key, vertex needs gcloud ADC, claude-code/codex need a logged-in local CLI), so
there is nothing safe to assume.
`chooseBackend` now fails clearly in disabled mode with no backend, naming the
(hidden) `--llm-backend` flag and its choices and noting each backend's
credential needs. `--llm-backend` stays hidden in `--help`, consistent with the
rest of the documented automation surface; the error message is the discovery
path.
- Add a unit test (no backend, disabled -> clear message) and a CLI/integration
test (`--target claude-code --no-input` -> exit 1, clear message, not the
Anthropic red herring).
- Document the no-default behavior and add a Common-errors row in
docs-site ktx-setup.mdx.
* refactor(cli): single source of truth for setup LLM backends
The set of LLM backends a user can pick during `ktx setup` (claude-code,
codex, anthropic, vertex) was hand-enumerated in five places: the
`--llm-backend` arg parser, the `KtxSetupLlmBackend` union, the interactive
prompt's narrowing, the prompt options, and the missing-backend error. Only
some had TypeScript coverage, so adding a backend could silently drift (e.g.
a valid value rejected by the parser, or routed to anthropic by the prompt's
`? : 'anthropic'` fallback).
Collapse them onto one `KTX_SETUP_LLM_BACKENDS` list:
- `KtxSetupLlmBackend` is derived from it.
- `isKtxSetupLlmBackend` is the shared validator; the arg parser and the
prompt both route through it instead of re-listing literals.
- The prompt options derive from the list, with a `Record<KtxSetupLlmBackend,
string>` label map so a new backend fails to compile until it has a label.
- The missing-backend error builds its choice list from the same source.
Behavior-preserving: identical accepted values and parse error, identical
prompt options (asserted by an existing test), and the prompt's unreachable
fallback now cancels rather than silently assuming anthropic.
2026-06-09 19:11:39 +02:00
With `--no-input`, `ktx setup` does not assume a default LLM provider, because
every backend needs credentials only you can supply. Pass `--llm-backend`
explicitly. Note that `--target` selects the agent integration, not the LLM
provider: `ktx setup --target claude-code --no-input` still needs
`--llm-backend claude-code` to use your Claude subscription for **ktx** LLM
calls.
2026-05-14 12:53:55 -04:00
### Embeddings
| Flag | Description |
|------|-------------|
| `--embedding-backend <backend>` | Embedding backend: `openai` or `sentence-transformers` |
| `--embedding-api-key-env <name>` | Environment variable containing the embedding provider API key |
| `--embedding-api-key-file <path>` | File containing the embedding provider API key |
| `--skip-embeddings` | Leave embedding setup incomplete |
2026-05-20 17:33:38 +02:00
`sentence-transformers` uses the **ktx**-managed Python runtime. Choose only one
2026-05-14 12:53:55 -04:00
embedding credential source.
2026-05-17 10:27:29 +02:00
### Runtime
Setup prepares the managed Python runtime when your selected configuration
2026-05-19 12:18:52 +02:00
needs it. In the full setup flow, the runtime step runs after database and
2026-05-20 17:33:38 +02:00
context-source setup and before the initial context build.
2026-05-19 12:18:52 +02:00
2026-05-20 17:33:38 +02:00
**ktx** prepares the `core` runtime feature when query-history ingest, Looker
context-source ingest, database introspection fallback, or daemon-backed
context build paths need it. **ktx** prepares the `local-embeddings` runtime feature when you
2026-05-19 12:18:52 +02:00
choose managed local `sentence-transformers` embeddings. Existing external
daemon URLs, such as `KTX_DAEMON_URL` or `KTX_SQL_ANALYSIS_URL`, satisfy the
matching dependency and skip managed runtime installation for that dependency.
`ktx setup --agents` doesn't prepare runtime features or build context. It only
installs agent configuration and rules. Start MCP with `ktx mcp start` before
using HTTP-based agents; MCP startup prepares the runtime it needs.
2026-05-17 10:27:29 +02:00
Interactive setup prompts before installing runtime features. Use `--yes` to
install them without prompting. Use `--no-input` to fail fast when required
runtime features are missing.
2026-05-14 12:53:55 -04:00
### Databases
| Flag | Description |
|------|-------------|
2026-05-22 14:22:11 +02:00
| `--database <driver>` | Database driver to configure; repeatable. Choices: `sqlite`, `postgres`, `mysql`, `clickhouse`, `sqlserver`, `bigquery`, `snowflake` |
2026-05-19 19:23:35 +02:00
| `--database-connection-id <id>` | Existing selected connection id; repeatable. With `--database` or `--database-url`, connection id for the new connection. |
2026-05-14 12:53:55 -04:00
| `--database-url <url>` | URL, `env:NAME`, or `file:/path` for one new URL-style database connection; also used as the SQLite path |
| `--database-schema <schema>` | Database schema or dataset to include; repeatable |
| `--skip-databases` | Leave database setup incomplete |
2026-05-20 17:33:38 +02:00
**ktx** needs at least one database connection before it can build database
2026-05-14 12:53:55 -04:00
context. Use `--skip-databases` only when intentionally leaving the project
incomplete.
2026-05-22 14:22:11 +02:00
`--database-schema` maps to the driver's scope field: `schemas` for PostgreSQL,
MySQL, and SQL Server; `schema_names` for Snowflake; `dataset_ids` for
BigQuery; and `databases` for ClickHouse.
2026-05-14 12:53:55 -04:00
### Query History
| Flag | Description |
|------|-------------|
| `--enable-query-history` | Enable query-history ingest when the selected database supports it |
| `--disable-query-history` | Disable query-history ingest for the selected database |
2026-05-15 15:31:51 -04:00
| `--query-history-window-days <number>` | BigQuery/Snowflake query-history lookback window |
2026-05-14 12:53:55 -04:00
| `--query-history-min-executions <number>` | Minimum executions for a query-history template |
| `--query-history-service-account-pattern <pattern>` | Query-history service-account regex; repeatable |
| `--query-history-redaction-pattern <pattern>` | Query-history SQL-literal redaction regex; repeatable |
2026-05-15 15:31:51 -04:00
Query history setup is supported for Postgres, BigQuery, and Snowflake. The
window flag applies to BigQuery and Snowflake; Postgres reads the current
`pg_stat_statements` aggregate data instead of a time-windowed history table.
2026-05-29 17:41:04 +02:00
Later `ktx ingest` runs build enriched context and need a configured model and
embeddings, including when query history is enabled.
2026-05-14 12:53:55 -04:00
2026-05-24 19:30:06 +02:00
When query history is enabled for PostgreSQL, Snowflake, or BigQuery,
`ktx setup` runs a non-blocking readiness probe after the connection test
passes. A failed probe still writes setup changes, prints the warehouse-specific
test: split cli tests from source tree (#216)
* feat(cli): define full warehouse dialect contract
* test(cli): keep dialect edge tests focused
* fix(cli): stabilize dialect contract foundation
* refactor(connectors): own read-only query preparation
* refactor(connectors): resolve dialects through registry
* refactor(connectors): keep concrete dialect classes internal
* chore(workspace): enforce dialect import boundary
* refactor(cli): resolve relationship dialect at scan boundary
* refactor(cli): use dialect display parsing for entity details
* refactor(cli): use dialect display parsing for warehouse catalog
* refactor(cli): use dialect SQL in relationship workflows
* test(cli): verify solid dialect scan workflow closure
* test: split cli tests from source tree
* refactor(cli): standardize BigQuery scope listing
* feat(sqlite): implement connector scope listing
* test(connectors): cover required table listing
* feat(cli): add warehouse driver registry
* refactor(setup): route scope discovery through driver registry
* refactor(cli): route local query execution through driver registry
* refactor(historic-sql): route dialect support through driver registry
* refactor(cli): test warehouse connections through driver registry
* fix(cli): close driver registry type export gaps
* Improve setup daemon diagnostics
* refactor(setup): centralize rail-prefixed diagnostics + query-history fallback
Extract errorMessage, writePrefixedLines, and flushPrefixedBufferedCommandOutput
into clack.ts so the setup wizard, managed daemons, and embedding/agent steps
share one rail-formatted writer. setup-databases.ts also adds a
"disable query history and retry" option when the schema-context build fails
and query history is the likely culprit, surfaced via a new
failed-query-history-unavailable status.
* fix(cli): carry catalog through the picker so BigQuery/Snowflake/SQL Server scope filters match
The setup picker's KtxTableListEntry was a 2-level { schema, name }, so
qualifiedTableId always wrote db.name into enabled_tables. When BigQuery,
Snowflake, or SQL Server later ran fast ingest, their introspect step filtered
the scope set with scopedTableNames(scope, { catalog: projectId|database, db })
— catalog was non-null on the introspect side but null in the scope refs, so
every entry was rejected, the live-database adapter staged zero table files,
and detect() failed with 'Adapter "live-database" did not recognize fetched
source output'.
Align the picker boundary with the canonical 3-level KtxTableRef:
- Add catalog: string | null to KtxTableListEntry.
- BigQuery/Snowflake/SQL Server listTables populate catalog from the
resolved projectId / database; Postgres/MySQL/ClickHouse/SQLite set null.
- qualifiedTableId emits catalog.schema.name when catalog is non-null
(resolveEnabledTables already accepts the 3-part shape) and
schemasFromEnabledTables now goes through parseDottedTableEntry so it
recovers the schema correctly from both 2-part and 3-part entries.
- Export parseDottedTableEntry from enabled-tables.ts (@internal) for picker
reuse.
Update listTables expectations in all seven connector tests and the setup /
picker test fixtures. Add a picker regression test that covers the
catalog-bearing round-trip (save + refine).
* fix(cli): allow debug telemetry under opt-out env
2026-05-26 08:49:05 +02:00
grant or extension remediation, and skips query-history processing until you
fix the prerequisite. If the later schema-context build also fails, interactive
setup offers **Disable query history and retry** so you can finish database
setup with `connections.<id>.context.queryHistory.enabled: false`.
2026-05-24 19:30:06 +02:00
2026-06-03 17:19:42 +02:00
After the schema scan completes, setup can derive query-history service-account
filters from in-scope history. If **ktx** finds clear operational roles, it
prints each proposed exclusion with a reason and writes
`connections.<id>.context.queryHistory.filters.serviceAccounts` only when you
apply the proposal. In non-interactive setup with `--yes`, the proposal is
applied automatically. Existing `serviceAccounts` blocks are never overwritten.
2026-05-24 19:30:06 +02:00
For BigQuery, the remediation tells you to grant `roles/bigquery.resourceViewer`
on the BigQuery project, or grant a custom role that contains
`bigquery.jobs.listAll`.
2026-05-14 12:53:55 -04:00
### Context Sources
2026-05-24 19:29:37 +02:00
In interactive setup, after you configure a database, choose
**Skip context sources** to leave optional context-source setup complete with no
sources. This is equivalent to passing `--skip-sources` in scripted setup.
2026-05-14 12:53:55 -04:00
| Flag | Description |
|------|-------------|
2026-05-20 17:33:38 +02:00
| `--source <type>` | Context-source connector type: `dbt`, `metricflow`, `metabase`, `looker`, `lookml`, or `notion` |
| `--source-connection-id <id>` | Connection id for context-source setup |
2026-05-14 12:53:55 -04:00
| `--source-path <path>` | Local source path for dbt, MetricFlow, or LookML |
| `--source-git-url <url>` | Git URL for dbt, MetricFlow, or LookML |
2026-05-20 17:33:38 +02:00
| `--source-branch <branch>` | Git branch for context-source setup |
| `--source-subpath <path>` | Repo subpath for context-source setup |
2026-05-29 17:23:46 +02:00
| `--source-auth-token-ref <ref>` | `env:` or `file:` credential reference for source repo auth or Notion integration token |
2026-05-14 12:53:55 -04:00
| `--source-url <url>` | Source service URL for Metabase or Looker |
2026-05-29 17:23:46 +02:00
| `--source-api-key-ref <ref>` | `env:` or `file:` API key reference for Metabase |
2026-05-14 12:53:55 -04:00
| `--source-client-id <id>` | Looker client id |
| `--source-client-secret-ref <ref>` | `env:` or `file:` Looker client secret reference |
2026-05-20 17:33:38 +02:00
| `--source-warehouse-connection-id <id>` | Warehouse connection id used for context-source mapping |
2026-05-14 12:53:55 -04:00
| `--source-project-name <name>` | dbt project name override |
| `--source-profiles-path <path>` | dbt profiles path |
2026-05-20 17:33:38 +02:00
| `--source-target <target>` | dbt target or context-source-specific mapping target |
2026-05-14 12:53:55 -04:00
| `--metabase-database-id <id>` | Metabase database id to map |
| `--notion-crawl-mode <mode>` | Notion crawl mode: `all_accessible` or `selected_roots` |
| `--notion-root-page-id <id>` | Notion root page id; repeatable |
2026-05-20 17:33:38 +02:00
| `--skip-sources` | Mark optional context-source setup complete with no sources |
2026-05-14 12:53:55 -04:00
Choose only one source location: `--source-path` or `--source-git-url`.
2026-05-11 00:45:43 -07:00
## Examples
```bash
# Run the interactive setup wizard
ktx setup
2026-05-13 17:55:25 -04:00
# Run setup for a specific project directory
ktx setup --project-dir ./analytics
2026-05-11 00:45:43 -07:00
2026-06-08 15:30:48 +02:00
# Use Claude Code for ktx LLM calls
2026-05-16 12:06:34 +02:00
ktx setup \
--project-dir ./analytics \
2026-06-08 15:30:48 +02:00
--llm-backend claude-code
2026-05-16 12:06:34 +02:00
feat: add codex llm backend for ktx runtime work (#253)
* feat: add codex sdk runner foundation
* feat: parse codex runtime events
* feat: expose codex runtime mcp tools
* feat: add codex llm runtime
* feat: wire codex llm backend
* test: avoid Array.fromAsync in codex runner test
* docs: document codex llm backend
* fix: tighten codex runtime config ownership
* fix: use codex sdk env and thread options
* fix: parse codex sdk event shapes
* test: add codex backend live smoke
* docs: clarify codex backend isolation
* fix: drive codex loop metrics from mcp events
* fix: enforce codex local step budget
* docs: disclose codex isolation limits
* fix: count all codex agent steps and stream step callbacks live
The agent-loop step budget only counted completed mcp_tool_call items, so
built-in command_execution steps (which the public Codex SDK/CLI surface can
still expose) never decremented the budget, letting ingest/reconciliation run
past stepBudget until Codex stopped on its own. onStepFinish was also replayed
only after the whole stream drained, so live work_unit_step / reconciliation
progress appeared stuck until the Codex process exited.
collectEvents is now the single live step accumulator: it counts every
completed agent-action item via a shared isCompletedAgentStep predicate
(command_execution, mcp_tool_call, file_change, web_search), fires onStepFinish
as each step completes, and enforces the budget on that broader count. A
no-tool turn still counts as one step. toolFailures stays MCP-specific, since a
non-zero command exit is normal agent exploration, not a loop failure.
* test: align ingest llm-guard assertions with codex backend
The skip-llm ingest guard message now lists codex as a valid backend and
mentions a Claude Code/Codex session plus a codex setup hint, but this slow
suite test still asserted the pre-codex wording. Update it to match the
production message (already covered by the local-bundle-runtime unit test) and
add the codex setup-line assertion.
* fix: treat codex error:null tool calls as success
The Codex SDK serializes error: null on successful mcp_tool_call items, so
the failure check (item.error !== undefined) flagged every successful tool
call as failed with the empty-payload default "Codex turn failed". This
killed every ingest work unit under the codex backend before it could
produce a patch.
Key on status === 'failed' (authoritative, always set) and only treat a
populated error object as a failure. Add a regression test built from a
verbatim real-SDK event capture.
* fix: default codex backend to gpt-5.5 and report real probe errors
The previous default gpt-5.3-codex is an API-key-only model that the OpenAI
API rejects under ChatGPT-account (subscription) auth, so codex status/setup
failed with a misleading "authentication is not usable" message even though
auth was fine.
- Default codex model is now gpt-5.5 (works on both subscription and API-key
auth); the curated setup picker offers gpt-5.5 / gpt-5.4 / gpt-5.4-mini and
keeps free-form entry for account-specific ids (e.g. gpt-5.3-codex-spark).
- runCodexAuthProbe now distinguishes "model not available" from an auth
failure and surfaces the real API error: collectEvents retains stream
events when the SDK throws on a non-zero exit, and the API error JSON
envelope is unwrapped to its human-readable message.
- The Codex isolation warning now renders inside the clack setup frame.
- Docs updated to gpt-5.5 with a note that *-codex ids require API-key auth.
* fix: require llm.models.default in status and match codex probe remediation
Status reported a project ready when a non-none LLM backend was configured
without llm.models.default, but the runtime (resolveModelSlots) hard-requires
it, so ingest/scan/memory threw after `ktx status` said the project was usable.
buildLlmStatus now fails for any non-none backend missing models.default and no
longer invents a fallback model for claude-code/codex.
Codex probe failures now carry a category-matched fix: a model-access failure
steers the user at llm.models.default instead of the auth/install remediation.
runCodexAuthProbe returns the fix and status consumes it; the message stays
self-sufficient so setup output is unchanged.
Docs: README now lists the codex backend and local Codex auth; ktx-setup.mdx
states --llm-model only accepts codex/default or gpt-*/codex-* ids.
Repaired four doctor fixtures that configured a backend without models.default
(the now-correctly-blocked config) and added coverage for the new behavior.
2026-06-02 13:57:11 +02:00
# Configure **ktx** to use local Codex authentication for LLM work
2026-06-08 15:30:48 +02:00
ktx setup --llm-backend codex --no-input
feat: add codex llm backend for ktx runtime work (#253)
* feat: add codex sdk runner foundation
* feat: parse codex runtime events
* feat: expose codex runtime mcp tools
* feat: add codex llm runtime
* feat: wire codex llm backend
* test: avoid Array.fromAsync in codex runner test
* docs: document codex llm backend
* fix: tighten codex runtime config ownership
* fix: use codex sdk env and thread options
* fix: parse codex sdk event shapes
* test: add codex backend live smoke
* docs: clarify codex backend isolation
* fix: drive codex loop metrics from mcp events
* fix: enforce codex local step budget
* docs: disclose codex isolation limits
* fix: count all codex agent steps and stream step callbacks live
The agent-loop step budget only counted completed mcp_tool_call items, so
built-in command_execution steps (which the public Codex SDK/CLI surface can
still expose) never decremented the budget, letting ingest/reconciliation run
past stepBudget until Codex stopped on its own. onStepFinish was also replayed
only after the whole stream drained, so live work_unit_step / reconciliation
progress appeared stuck until the Codex process exited.
collectEvents is now the single live step accumulator: it counts every
completed agent-action item via a shared isCompletedAgentStep predicate
(command_execution, mcp_tool_call, file_change, web_search), fires onStepFinish
as each step completes, and enforces the budget on that broader count. A
no-tool turn still counts as one step. toolFailures stays MCP-specific, since a
non-zero command exit is normal agent exploration, not a loop failure.
* test: align ingest llm-guard assertions with codex backend
The skip-llm ingest guard message now lists codex as a valid backend and
mentions a Claude Code/Codex session plus a codex setup hint, but this slow
suite test still asserted the pre-codex wording. Update it to match the
production message (already covered by the local-bundle-runtime unit test) and
add the codex setup-line assertion.
* fix: treat codex error:null tool calls as success
The Codex SDK serializes error: null on successful mcp_tool_call items, so
the failure check (item.error !== undefined) flagged every successful tool
call as failed with the empty-payload default "Codex turn failed". This
killed every ingest work unit under the codex backend before it could
produce a patch.
Key on status === 'failed' (authoritative, always set) and only treat a
populated error object as a failure. Add a regression test built from a
verbatim real-SDK event capture.
* fix: default codex backend to gpt-5.5 and report real probe errors
The previous default gpt-5.3-codex is an API-key-only model that the OpenAI
API rejects under ChatGPT-account (subscription) auth, so codex status/setup
failed with a misleading "authentication is not usable" message even though
auth was fine.
- Default codex model is now gpt-5.5 (works on both subscription and API-key
auth); the curated setup picker offers gpt-5.5 / gpt-5.4 / gpt-5.4-mini and
keeps free-form entry for account-specific ids (e.g. gpt-5.3-codex-spark).
- runCodexAuthProbe now distinguishes "model not available" from an auth
failure and surfaces the real API error: collectEvents retains stream
events when the SDK throws on a non-zero exit, and the API error JSON
envelope is unwrapped to its human-readable message.
- The Codex isolation warning now renders inside the clack setup frame.
- Docs updated to gpt-5.5 with a note that *-codex ids require API-key auth.
* fix: require llm.models.default in status and match codex probe remediation
Status reported a project ready when a non-none LLM backend was configured
without llm.models.default, but the runtime (resolveModelSlots) hard-requires
it, so ingest/scan/memory threw after `ktx status` said the project was usable.
buildLlmStatus now fails for any non-none backend missing models.default and no
longer invents a fallback model for claude-code/codex.
Codex probe failures now carry a category-matched fix: a model-access failure
steers the user at llm.models.default instead of the auth/install remediation.
runCodexAuthProbe returns the fix and status consumes it; the message stays
self-sufficient so setup output is unchanged.
Docs: README now lists the codex backend and local Codex auth; ktx-setup.mdx
states --llm-model only accepts codex/default or gpt-*/codex-* ids.
Repaired four doctor fixtures that configured a backend without models.default
(the now-correctly-blocked config) and added coverage for the new behavior.
2026-06-02 13:57:11 +02:00
```
When you choose `--llm-backend codex`, setup prints a warning if the public
Codex SDK and CLI surface cannot prove full Claude-Code-style isolation. The
backend restricts **ktx** runtime MCP tools to each run, but Codex may still
load user Codex config and built-in command execution or read-only file
capabilities.
```bash
2026-05-14 12:53:55 -04:00
# Script a Postgres connection that reads its URL from the environment
ktx setup \
--project-dir ./analytics \
--no-input \
2026-05-19 19:23:35 +02:00
--yes \
2026-05-14 12:53:55 -04:00
--skip-llm \
--skip-embeddings \
--database postgres \
2026-05-19 19:23:35 +02:00
--database-connection-id warehouse \
2026-05-14 12:53:55 -04:00
--database-url env:DATABASE_URL \
--database-schema public
# Enable Postgres query history while setting up a database
ktx setup \
--project-dir ./analytics \
--database postgres \
2026-05-19 19:23:35 +02:00
--database-connection-id warehouse \
2026-05-14 12:53:55 -04:00
--database-url env:DATABASE_URL \
--enable-query-history \
--query-history-min-executions 5
# Add a Metabase source mapped to an existing warehouse connection
ktx setup \
--source metabase \
--source-connection-id prod_metabase \
--source-url https://metabase.example.com \
--source-api-key-ref env:METABASE_API_KEY \
--source-warehouse-connection-id warehouse \
--metabase-database-id 1
2026-05-29 17:23:46 +02:00
# Add a Notion source that crawls selected root pages
ktx setup \
--source notion \
--source-connection-id notion-main \
--source-auth-token-ref env:NOTION_TOKEN \
--notion-crawl-mode selected_roots \
--notion-root-page-id abc123def456
2026-05-14 12:53:55 -04:00
# Install project-scoped agent integration for Codex
ktx setup --agents --target codex
2026-05-11 00:45:43 -07:00
```
2026-05-11 16:43:08 -07:00
## Output
2026-05-13 00:38:26 +02:00
Interactive setup renders prompts and progress messages. Use `ktx status` to
check setup and context readiness after setup exits.
2026-05-11 16:43:08 -07:00
```text
2026-05-20 17:33:38 +02:00
ktx project: /home/user/analytics
2026-05-11 16:43:08 -07:00
Project ready: yes
LLM ready: yes (claude-sonnet-4-6)
Embeddings ready: yes (text-embedding-3-small)
2026-05-14 01:43:06 +02:00
Databases configured: yes (postgres-warehouse)
2026-05-11 16:43:08 -07:00
Context sources configured: yes (dbt-main)
2026-05-17 10:27:29 +02:00
Runtime ready: yes (core)
2026-05-20 17:33:38 +02:00
ktx context built: yes
2026-05-11 16:43:08 -07:00
Agent integration ready: yes (codex:project)
```
2026-05-14 12:53:55 -04:00
Use `ktx status` for repeatable readiness checks after setup exits.
2026-05-11 16:43:08 -07:00
## Common errors
| Error | Cause | Recovery |
|-------|-------|----------|
| Setup resumes an unexpected project | `KTX_PROJECT_DIR` or nearest `ktx.yaml` points to another directory | Pass `--project-dir <path>` explicitly |
2026-05-14 12:53:55 -04:00
| Setup cannot run in CI | Required values are missing and `--no-input` disables prompts | Provide the relevant automation flags or create a fixture `ktx.yaml` |
fix(cli): clear error when ktx setup has no LLM backend under --no-input (#281)
* fix(cli): fail clearly when ktx setup has no LLM backend under --no-input
Non-interactive `ktx setup` silently defaulted the LLM backend to `anthropic`
and then failed with `Missing Anthropic API key: pass --anthropic-api-key-env
or --anthropic-api-key-file` — confusing for users who selected a different
provider (e.g. `--target claude-code`) and never asked for the Anthropic API
backend.
That silent default could never succeed: it was reached only when no backend,
Anthropic key, or Vertex flag was supplied, and in exactly that case the
Anthropic credential resolver always failed (no env fallback in disabled mode).
Unlike embeddings, the LLM has no credential-free default (anthropic needs a
key, vertex needs gcloud ADC, claude-code/codex need a logged-in local CLI), so
there is nothing safe to assume.
`chooseBackend` now fails clearly in disabled mode with no backend, naming the
(hidden) `--llm-backend` flag and its choices and noting each backend's
credential needs. `--llm-backend` stays hidden in `--help`, consistent with the
rest of the documented automation surface; the error message is the discovery
path.
- Add a unit test (no backend, disabled -> clear message) and a CLI/integration
test (`--target claude-code --no-input` -> exit 1, clear message, not the
Anthropic red herring).
- Document the no-default behavior and add a Common-errors row in
docs-site ktx-setup.mdx.
* refactor(cli): single source of truth for setup LLM backends
The set of LLM backends a user can pick during `ktx setup` (claude-code,
codex, anthropic, vertex) was hand-enumerated in five places: the
`--llm-backend` arg parser, the `KtxSetupLlmBackend` union, the interactive
prompt's narrowing, the prompt options, and the missing-backend error. Only
some had TypeScript coverage, so adding a backend could silently drift (e.g.
a valid value rejected by the parser, or routed to anthropic by the prompt's
`? : 'anthropic'` fallback).
Collapse them onto one `KTX_SETUP_LLM_BACKENDS` list:
- `KtxSetupLlmBackend` is derived from it.
- `isKtxSetupLlmBackend` is the shared validator; the arg parser and the
prompt both route through it instead of re-listing literals.
- The prompt options derive from the list, with a `Record<KtxSetupLlmBackend,
string>` label map so a new backend fails to compile until it has a label.
- The missing-backend error builds its choice list from the same source.
Behavior-preserving: identical accepted values and parse error, identical
prompt options (asserted by an existing test), and the prompt's unreachable
fallback now cancels rather than silently assuming anthropic.
2026-06-09 19:11:39 +02:00
| `Missing LLM backend: pass --llm-backend …` | `--no-input` setup ran without an LLM backend; `--target` does not select one | Pass `--llm-backend claude-code`, `codex`, `anthropic`, or `vertex` (with that backend's credential flags) |
2026-05-14 12:53:55 -04:00
| Provider health check fails | Provider key, model id, Vertex project, or Vertex location is invalid | Fix the `env:` or `file:` reference and rerun setup |
2026-05-20 01:36:54 +02:00
| Python runtime is missing | The selected setup needs runtime-backed agent, query-history, Looker, or local embedding features | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx admin runtime install` command |
2026-05-14 12:53:55 -04:00
| `--enable-query-history` is rejected | The selected database driver does not support query history | Use Postgres, BigQuery, or Snowflake, or rerun without query-history flags |
| Source setup rejects location flags | Both `--source-path` and `--source-git-url` were supplied | Choose the local path or the Git URL, not both |
2026-05-12 23:51:46 +02:00
| Agent integration missing | Setup skipped the agents step | Run `ktx setup --agents --target <target>` |
2026-05-19 19:23:35 +02:00
| Agent setup cannot prompt for a target | Non-TTY `ktx setup --agents` needs a target | Run `ktx setup --agents --target <target>` or rerun in a TTY |
2026-05-14 12:53:55 -04:00
| Global agent install is rejected | `--global` was used with a target other than `claude-code` or `codex` | Omit `--global`, or choose `--target claude-code` or `--target codex` |