Compare commits

...

92 commits
v0.7.0 ... main

Author SHA1 Message Date
Andrey Avtomonov
4e61020089
feat(cli): let ktx setup --agents choose an install directory (#298)
Split the fused directory concept into projectDir (what the agent config
references) and installRoot (where project-scoped files are written), so
users can install .claude/, .mcp.json, skills, and rules where they open
their agent instead of only in the ktx project directory.

- Add --install-dir <path> (resolved against cwd, created if missing,
  mutually exclusive with --global/--local, rejected for claude-desktop).
- Add an interactive directory menu: ktx project dir / Current directory
  (hidden when it equals the project dir) / Custom directory… / Global
  scope (shown only when every target supports it).
- Expand a leading ~ in typed/quoted paths so the ~/… menu hints round-trip.
- Record installRoot in the install manifest and merge key; thread it
  through file planning, MCP config paths, summaries, and next actions.
- Refresh uv.lock to 0.12.0 for the editable ktx-sl and ktx-daemon packages.
2026-06-13 00:46:56 +02:00
github-actions[bot]
ed44f46f2a chore: refresh star history chart [skip ci] 2026-06-12 18:46:09 +00:00
semantic-release-bot
cf2871ec8b chore(release): 0.12.0 [skip ci]
## [0.12.0](https://github.com/Kaelio/ktx/compare/v0.11.0...v0.12.0) (2026-06-12)

### Features

* **cli:** add ktx wordmark banner to setup intro ([#290](https://github.com/Kaelio/ktx/issues/290)) ([28953eb](28953eb616))
* **cli:** self-provision pinned uv and defer MCP Python runtime install ([#297](https://github.com/Kaelio/ktx/issues/297)) ([feb0818](feb0818444))
* **cli:** setup progress spinners, Tab-to-select, and banner polish ([#296](https://github.com/Kaelio/ktx/issues/296)) ([663eaff](663eaff940)), closes [#FF8A4C](https://github.com/Kaelio/ktx/issues/FF8A4C)

### Bug Fixes

* classify MCP SQL query errors as expected ([#285](https://github.com/Kaelio/ktx/issues/285)) ([036a745](036a745fc1))
* **cli:** clear error when ktx setup has no LLM backend under --no-input ([#281](https://github.com/Kaelio/ktx/issues/281)) ([0425160](0425160857))
* **cli:** isolate ktx-owned project repositories ([#283](https://github.com/Kaelio/ktx/issues/283)) ([2877b85](2877b85adc))
* **cli:** own a dedicated git repo at the project dir when nested in an enclosing repo ([#282](https://github.com/Kaelio/ktx/issues/282)) ([fd18caa](fd18caa26a))
* **cli:** survive ktx.yaml version skew and derive repo ownership from disk ([#293](https://github.com/Kaelio/ktx/issues/293)) ([0689d70](0689d709d2))
* **deps:** bump hono override to 4.12.21 to resolve dependabot alerts ([#288](https://github.com/Kaelio/ktx/issues/288)) ([56e0633](56e06334d2))
* **ingest:** verify repair outcomes and reject dangling join targets ([#292](https://github.com/Kaelio/ktx/issues/292)) ([a278d2f](a278d2f7d0))
* read semantic sources safely ([#284](https://github.com/Kaelio/ktx/issues/284)) ([f3f893b](f3f893bf01))
* **setup:** require explicit no-input database scope ([#286](https://github.com/Kaelio/ktx/issues/286)) ([853f39a](853f39a7c3))

### Documentation

* **integrations:** correct context-source ingestion details ([#291](https://github.com/Kaelio/ktx/issues/291)) ([7c3b4ce](7c3b4cea2c))
* **site:** relocate GitHub stars to sidebar footer, add light/dark switcher ([#294](https://github.com/Kaelio/ktx/issues/294)) ([e1067bf](e1067bf734))

### Code Refactoring

* enforce ktx naming and AGENTS.md compliance sweep ([#289](https://github.com/Kaelio/ktx/issues/289)) ([00cdf2d](00cdf2de90))

### Tests

* **ingest:** supply explicit no-input schema scope in skip-llm setup ([#287](https://github.com/Kaelio/ktx/issues/287)) ([058051f](058051f1b9)), closes [#286](https://github.com/Kaelio/ktx/issues/286) [#286](https://github.com/Kaelio/ktx/issues/286)

### Other Changes

* refresh star history chart [skip ci] ([005c5fc](005c5fc860))
* refresh star history chart [skip ci] ([b076431](b076431b0a))
* refresh star history chart [skip ci] ([65de75e](65de75ebd7))
* remove dead pnpm.onlyBuiltDependencies from package.json ([9ff0e86](9ff0e86bb8))
2026-06-12 16:45:18 +00:00
Andrey Avtomonov
feb0818444
feat(cli): self-provision pinned uv and defer MCP Python runtime install (#297)
Fixes a production crash-loop (PostHog issue 019eb68e): ktx mcp start
--foreground on a uv-less container eagerly installed the managed Python
runtime at boot, failed, and was restarted by its supervisor every ~62s
(122 exceptions from one install).

- MCP server factory now wires a lazy semantic-layer compute port that
  defers the runtime install to the first call, mirroring the already-lazy
  SQL-analysis port; the server boots and serves non-Python tools without
  the runtime.
- ktx no longer requires uv on PATH: it downloads its own pinned,
  sha256-verified uv build under the runtime root (KTX_RUNTIME_ROOT aware),
  always musl-static on Linux. PATH uv is never consulted.
- uv is acquired before the version dir is wiped, so a failed download
  cannot destroy an existing runtime.
- Acquisition failures (offline, intercepted download, unsupported
  platform) throw KtxExpectedError and stay out of Error Tracking; a
  missing binary inside a checksum-verified archive remains a plain Error.
- scripts/refresh-uv-manifest.mjs regenerates the pinned manifest
  (packages/cli/src/managed-uv-release.ts) on uv bumps.
- Setup consent prompt now discloses the uv download; docs updated.
2026-06-12 16:31:06 +00:00
Andrey Avtomonov
663eaff940
feat(cli): setup progress spinners, Tab-to-select, and banner polish (#296)
* fix(cli): double the height of the setup banner t crossbar

* fix(cli): unify setup multi-select hints and make Tab the select key

The six interactive multi-select surfaces in `ktx setup` documented three
different hint voices, one had no hint at all, and they named two different
select keys (Space vs Tab). Tab is the only key that can toggle selection
without colliding with type-to-search input, so make it the single documented
select key everywhere and compose every hint from one shared fragment
vocabulary in prompt-navigation.ts.

- Register `updateSettings({ aliases: { tab: 'space' } })` so Tab toggles flat
  multiselects; the alias applies only to non-text prompts, leaving typed
  search input (schema/Notion) untouched.
- Add the missing hint to the agent-targets prompt and drop the stray
  "Space to select … Esc …" info line plus the now-dead writeSetupInfo helper.
- Replace the schema-scope ad-hoc hint with the searchable-multiselect voice
  and standardize "filter" -> "search" vocabulary.
- Delete DEFAULT_TREE_PICKER_HELP_TEXT and the unused TreePickerChrome.helpText
  seam; render the shared tree hint instead.

* refactor(cli): show LLM check progress for every setup backend

Rename runLlmHealthCheckWithProgress to validateModelWithProgress and
wrap the Claude subscription and Codex auth probes in the same spinner
progress as the Anthropic API and Vertex backends, so each backend shows
consistent "Checking <provider> LLM" output during setup.

* feat(cli): add ktx-orange progress spinners to setup steps

Add a shared runWithCliSpinner helper and a TTY-aware createCliSpinner:
an animated clack spinner in a terminal, and a static stderr-only spinner
before raw-mode pickers (the table tree picker and demo tour), where the
animated spinner's stdin grab would otherwise corrupt the next prompt.

Wrap the slow setup waits in progress spinners: managed runtime install,
embedding daemon start + first-run model download, embeddings health
check, the connection-test gate, and source validation / dbt clone /
Metabase discovery. Recolor every spinner frame from clack's magenta to
the ktx mascot orange (#FF8A4C) via the static helper and clack's
styleFrame option.
2026-06-12 16:43:10 +02:00
Andrey Avtomonov
e1067bf734
docs(site): relocate GitHub stars to sidebar footer, add light/dark switcher (#294)
Move the live GitHub stars widget into the sidebar footer pill as a
type:"icon" link, sitting opposite the Slack mark (space-between) and
beside the theme switcher. Render it as inner content (GitHub mark +
star + compact count) so fumadocs supplies the anchor.

Replace the default fumadocs theme switcher with a custom two-icon
control where each icon selects its own theme. The built-in
"light-dark" mode is a single blind toggle that flips on any click, so
clicking the sun while already in light mode jumps to dark. useTheme is
sourced from fumadocs-ui/provider/base and the icons are inlined to
avoid bare next-themes / lucide-react imports.
2026-06-12 09:06:04 +00:00
Andrey Avtomonov
0689d709d2
fix(cli): survive ktx.yaml version skew and derive repo ownership from disk (#293)
* fix(cli): survive ktx.yaml version skew and derive repo ownership from disk

Loading ktx.yaml is now tolerant of keys this ktx version does not
recognize: they are stripped from the in-memory config (the file on disk
is never rewritten) and reported by ktx status as non-blocking warnings,
while invalid values on recognized fields still fail hard. Repo
ownership is derived from observed state (a .git directory plus a root
ktx.yaml) instead of a ktx.managed git-config marker, so projects
created by any past or future ktx classify identically. initKtxProject
now runs an explicit foreign-repo pre-check and writes ktx.yaml before
initializing git, so an interrupted init leaves only recoverable
residue instead of a bare .git misread as foreign.

* style(cli): trim comment blocks to constraint-only notes

* docs(agents): require constraint-only code comments
2026-06-11 22:10:47 +02:00
Andrey Avtomonov
a278d2f7d0
fix(ingest): verify repair outcomes and reject dangling join targets (#292)
One ingest integration hiccup no longer discards a whole source:

- Replace the duplicated gate-repair and textual-resolver loops with one
  shared constrained-repair loop whose success criterion is re-running
  the failed check (verify), not whether the agent edited files. Verify
  failures feed the retry prompt; maxAttempts is 2.
- Let the resolver declare a conflicting patch redundant: a verified
  no-change resolution is accepted as subsumed instead of failing the
  source (duplicate wiki-page creation from parallel work units).
- Carry per-source validation errors through validateWuTouchedSources
  into gate messages and work-unit failure reasons instead of
  discarding them.
- Move join-neighbor expansion into the shared validation path so
  work-unit validation and integration gates check the same set.
- Reject joins whose target resolves to no source, at sl_write time and
  in the gates, attributed to the declaring source. Resolution mirrors
  the Python engine exactly (case-sensitive name within the
  connection), with a case-mismatch hint for the writing agent.
2026-06-11 14:39:51 +02:00
Andrey Avtomonov
00cdf2de90
refactor: enforce ktx naming and AGENTS.md compliance sweep (#289)
Align the tree with AGENTS.md/CLAUDE.md conventions:

- Rewrite user-facing strings, docs, and tests to lowercase `ktx`
  (no bare uppercase `KTX` tokens remain outside literal identifiers).
- Drop the legacy `historicSql` migration path and its now-unused
  helpers, per the no-backward-compat rule.
- Remove `as unknown as` / `any` casts: narrow `BaseTool` generics to
  `z.ZodObject`, add a typed `createLookerClient`, and delete the dead
  `getParametersSchema`/`toAnthropicFormat` pre-AI-SDK helpers.
- Use `InvalidArgumentError` for Commander parse failures.
- Finish the adapter→connector prose conversion in the `ktx.yaml` docs
  while keeping the literal `adapters` config key.
2026-06-11 13:49:45 +02:00
github-actions[bot]
005c5fc860 chore: refresh star history chart [skip ci] 2026-06-11 07:49:11 +00:00
Luca Martial
7c3b4cea2c
docs(integrations): correct context-source ingestion details (#291)
Verified the dbt, MetricFlow, LookML, Metabase, Looker, and Notion
sections of context-sources.mdx against the adapter code and fixed
claims that did not match the implementation:

- dbt: replace "test coverage" framing with the actual constraint/enum/
  join derivation; name both overlay and wiki outputs; fix work-unit
  granularity (per models/ schema file above 25 YAML files).
- MetricFlow: relationships come from entities (not dimensions); surface
  the join edges they produce.
- LookML: chunking is one work unit per model (not connected component);
  add the wiki output; note that a connection: mismatch disables SL writes.
- Metabase: dashboards are never fetched (no dashboard endpoint); work
  units are per collection; "usage patterns" is really card output schema.
- Looker: drop invented "purpose/audience" framing; describe triage as a
  prioritization gate; include Looks alongside explores and dashboards.
- Notion: not knowledge-only (it writes SL sources for mapped non-Notion
  targets); remove the nonexistent database-schema extraction; reframe
  "What it provides" as inputs; document root_data_source_ids.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 10:14:47 -07:00
Andrey Avtomonov
28953eb616
feat(cli): add ktx wordmark banner to setup intro (#290)
Render a lowercase ktx half-block wordmark with the brand-orange gradient
above the `ktx setup` intro on interactive TTYs. The banner degrades
through truecolor, xterm-256, and monochrome, and is skipped on non-TTY,
non-Unicode, or too-narrow terminals.

Extract shared color/Unicode capability detection into io helpers
(shouldUseColorOutput, colorDepthForOutput, unicodeSupported) so the
banner and doctor report route through one implementation.
2026-06-10 14:47:34 +00:00
Andrey Avtomonov
56e06334d2
fix(deps): bump hono override to 4.12.21 to resolve dependabot alerts (#288) 2026-06-10 12:26:01 +00:00
Andrey Avtomonov
058051f1b9
test(ingest): supply explicit no-input schema scope in skip-llm setup (#287)
The "prints provider setup guidance when a skip-llm setup project runs
ingest" test drives runKtxSetup in --no-input (inputMode: disabled) mode
with a postgres warehouse but databaseSchemas: []. PR #286 changed the
no-input contract to require an explicit database scope instead of
auto-scanning, so setup now exits 1 and the test's first assertion
(resolves.toBe(0)) fails. #286 updated setup-databases.test.ts but missed
this runKtxSetup call in ingest.test.ts. Pass databaseSchemas: ['public']
to satisfy the new contract, matching #286's own test updates.
2026-06-10 12:22:22 +00:00
Andrey Avtomonov
2877b85adc
fix(cli): isolate ktx-owned project repositories (#283)
* fix(cli): isolate ktx project git repos

* fix(cli): remove inert auto commit config

* test(cli): drop stale auto commit fixtures

* docs: document isolated ktx project repos

* test(cli): keep stale config grep clean

* fix(cli): guide setup away from foreign repos at the project dir

ktx owns the git repo rooted at the project dir and refuses to adopt one it
did not create (the Finding 3 isolation invariant). But setup steered users
straight into that failure: the interactive menu offers "Current directory"
first, and `--no-input --yes --project-dir <repo-root>` created directly in
place — both then threw a generic "Failed to initialize git repository:"
wrapper from deep in GitService.initialize().

Extract the ownership rule into a shared `classifyKtxRepoOwnership(dir)` used by
both GitService.initialize() (the invariant) and the setup wizard (pre-flight
guidance), so the decision derives from one rule. Setup now detects a foreign
repo before constructing GitService and: interactively re-prompts (the user
picks the existing `ktx-project` subfolder), or non-interactively returns a
clean missing-input with the actionable message. The typed foreign-repo error
is also surfaced verbatim instead of being buried under the generic wrapper.

Empty/non-repo current directories still work — only foreign repos are blocked.

* fix(cli): keep classifyKtxRepoOwnership total for non-directory paths

The setup ownership guard runs before the existing not-a-directory check, so
pointing a custom/--project-dir path at a file made classifyKtxRepoOwnership
lstat `<file>/.git`, hit ENOTDIR, and throw — crashing the setup step instead
of returning the friendly "path exists and is not a directory" result.

A path that is a file (or missing) holds no git repo for ktx to avoid, so treat
ENOTDIR like ENOENT and return 'unowned'. The downstream existingFolderState
check still rejects a non-directory with its friendly message, and the
classifier no longer throws raw errno for any caller.
2026-06-10 14:12:25 +02:00
Andrey Avtomonov
f3f893bf01
fix: read semantic sources safely (#284)
* fix: read semantic sources safely

* test: retarget reindex per-scope error case to a broken manifest

Reading a broken standalone source was made non-fatal in de1f1a8d (it is
surfaced for repair instead of throwing), so the reindex per-scope error
test no longer captured an error. Point it at a corrupt manifest shard,
which is the remaining fatal read failure the per-scope catch must
isolate, and assert the captured error names the offending file.

* fix(sl): decouple semantic-layer file names from warehouse naming rules

The in-file `name:` field is now the sole source identity; the filename is
a derived label that never participates in identity. This removes the
"Unsafe semantic-layer source name" failure class entirely: any warehouse
identifier (Snowflake's uppercase SIGNED_UP, EVENT$LOG, dotted names) can
be read, overlaid, edited, and deleted.

- New `source-files.ts`: one total filename derivation (safe lowercase
  names verbatim; otherwise slug + sha256-hash suffix, immune to
  case-insensitive-filesystem collisions) and one by-name file resolver.
- Reads resolve by name everywhere; the path-from-name fast path and
  `assertSafeSourceName` are gone.
- Writes resolve-then-write: rewrites land on the file that declares the
  name (human renames survive); new sources get a derived filename; a
  derived path occupied by a different source fails instead of clobbering.
- `readSourceFile` returns null for missing files instead of forcing every
  caller to launder IO errors; `deleteSource` distinguishes manifest-backed
  sources from not-found instead of silently succeeding.
- `sl_write_source` accepts verbatim warehouse identifiers (snake_case is
  now a recommendation for new sources) and rejects sourceName/source.name
  mismatches; `sl_edit_source` rejects name-changing edits.
- Ingest projection commits, gate-repair allowlists, and touched-source
  derivation use resolved paths / in-file names instead of interpolating
  `<connId>/<name>.yaml`.
- Collapsed the five parallel path derivations and duplicated path-token
  helpers onto the shared module; dropped dead service methods.

* fix(sl): resolve sources by declared name end-to-end and gate warehouse SQL with the parser-backed validator

- Key broken/renamed semantic-layer files by their recoverable in-file
  name (slSourceNameForFile) so mid-edit sources stay reachable under
  their real identity in reads, listings, and search
- Derive finalization touched sources from composed-source diffs and
  recover deleted files' declared names from the pre-change commit
  instead of parsing hash-derived filenames
- Resolve revert/rollback paths against history (listFilesAtCommit) so
  human-renamed files are restored where they lived at preHead
- Validate ingest sql_execution through the daemon's sqlglot
  validateReadOnly in the connection's dialect, sharing one
  driver-to-dialect map (sql-analysis/dialect.ts) across MCP and ingest
- Harden the local read-only SQL backstop: accept leading comments,
  reject smuggled second statements, and strip trailing
  semicolons/comments before row-limit wrapping
2026-06-10 14:06:13 +02:00
Andrey Avtomonov
853f39a7c3
fix(setup): require explicit no-input database scope (#286)
* test(setup): supply explicit --no-input scope to disabled-mode database tests

* fix(setup): require explicit database scope in --no-input instead of auto-scanning the warehouse

* docs(setup): document --no-input database scope requirement
2026-06-10 10:36:53 +00:00
Andrey Avtomonov
036a745fc1
fix: classify MCP SQL query errors as expected (#285) 2026-06-10 11:42:31 +02:00
github-actions[bot]
b076431b0a chore: refresh star history chart [skip ci] 2026-06-10 07:36:58 +00:00
Andrey Avtomonov
fd18caa26a
fix(cli): own a dedicated git repo at the project dir when nested in an enclosing repo (#282)
GitService.initialize() used checkIsRepo(), which is true whenever the project
dir sits anywhere inside a git working tree. So when a ktx project lived in a
subdirectory of an enclosing repo, ktx skipped `git init` and silently adopted
the enclosing repo as its store.

Every ktx relative path assumes the project dir IS the working-tree root. During
ingest, wiki/SL pages are written through a session worktree (whose root is the
worktree dir, so the page is recorded at repo-relative `wiki/global/<key>.md`)
and then squash-merged into the main worktree. With an adopted enclosing repo,
the main worktree's root is the enclosing git root, so the merge wrote the page
to `<gitRoot>/wiki/global/` — outside the project dir. reindex scans
`<projectDir>/wiki/global/`, found nothing, and wiki_search silently returned
empty (knowledge_pages = 0) even though ingest reported success.

Detect the project dir's own root with checkIsRepo(IS_REPO_ROOT) and initialize
a dedicated repo there unless the project dir is already a repo root. This keeps
adopting a user-created repo when the project dir IS that repo's root, fixes the
silent wiki/SL/memory divergence at its source for every writer, and stops ktx
from committing its scaffold into the user's enclosing repo.

Regression tests cover both layers: a project nested in an enclosing repo gets
its own .git (and the enclosing repo stays untouched), and a wiki page written
through a session worktree + squash-merge lands in the project dir and is
discovered by reindex.
2026-06-09 23:37:24 +02:00
github-actions[bot]
65de75ebd7 chore: refresh star history chart [skip ci] 2026-06-09 18:49:48 +00:00
Andrey Avtomonov
0425160857
fix(cli): clear error when ktx setup has no LLM backend under --no-input (#281)
* fix(cli): fail clearly when ktx setup has no LLM backend under --no-input

Non-interactive `ktx setup` silently defaulted the LLM backend to `anthropic`
and then failed with `Missing Anthropic API key: pass --anthropic-api-key-env
or --anthropic-api-key-file` — confusing for users who selected a different
provider (e.g. `--target claude-code`) and never asked for the Anthropic API
backend.

That silent default could never succeed: it was reached only when no backend,
Anthropic key, or Vertex flag was supplied, and in exactly that case the
Anthropic credential resolver always failed (no env fallback in disabled mode).
Unlike embeddings, the LLM has no credential-free default (anthropic needs a
key, vertex needs gcloud ADC, claude-code/codex need a logged-in local CLI), so
there is nothing safe to assume.

`chooseBackend` now fails clearly in disabled mode with no backend, naming the
(hidden) `--llm-backend` flag and its choices and noting each backend's
credential needs. `--llm-backend` stays hidden in `--help`, consistent with the
rest of the documented automation surface; the error message is the discovery
path.

- Add a unit test (no backend, disabled -> clear message) and a CLI/integration
  test (`--target claude-code --no-input` -> exit 1, clear message, not the
  Anthropic red herring).
- Document the no-default behavior and add a Common-errors row in
  docs-site ktx-setup.mdx.

* refactor(cli): single source of truth for setup LLM backends

The set of LLM backends a user can pick during `ktx setup` (claude-code,
codex, anthropic, vertex) was hand-enumerated in five places: the
`--llm-backend` arg parser, the `KtxSetupLlmBackend` union, the interactive
prompt's narrowing, the prompt options, and the missing-backend error. Only
some had TypeScript coverage, so adding a backend could silently drift (e.g.
a valid value rejected by the parser, or routed to anthropic by the prompt's
`? : 'anthropic'` fallback).

Collapse them onto one `KTX_SETUP_LLM_BACKENDS` list:
- `KtxSetupLlmBackend` is derived from it.
- `isKtxSetupLlmBackend` is the shared validator; the arg parser and the
  prompt both route through it instead of re-listing literals.
- The prompt options derive from the list, with a `Record<KtxSetupLlmBackend,
  string>` label map so a new backend fails to compile until it has a label.
- The missing-backend error builds its choice list from the same source.

Behavior-preserving: identical accepted values and parse error, identical
prompt options (asserted by an existing test), and the prompt's unreachable
fallback now cancels rather than silently assuming anthropic.
2026-06-09 17:11:39 +00:00
Andrey Avtomonov
9ff0e86bb8 chore: remove dead pnpm.onlyBuiltDependencies from package.json
pnpm 11 no longer reads the package.json "pnpm" field and warns on it.
The build allowlist already lives in pnpm-workspace.yaml via allowBuilds
(better-sqlite3, esbuild, sharp), so this block was redundant noise.
2026-06-09 17:24:43 +02:00
semantic-release-bot
07ab275662 chore(release): 0.11.0 [skip ci]
## [0.11.0](https://github.com/Kaelio/ktx/compare/v0.10.0...v0.11.0) (2026-06-09)

### Features

* **cli:** add Slack community CTA on errors, crashes, setup, and help ([#277](https://github.com/Kaelio/ktx/issues/277)) ([66517fc](66517fc320))

### Bug Fixes

* **cli:** classify ktx setup abandonment as aborted, not a blank error ([#278](https://github.com/Kaelio/ktx/issues/278)) ([470802e](470802e58e))
* **cli:** ensure git committer identity during ktx setup ([#276](https://github.com/Kaelio/ktx/issues/276)) ([6b2f7c3](6b2f7c3365))

### Documentation

* **agents:** sync Opinionated Product Defaults guidance into AGENTS.md ([#280](https://github.com/Kaelio/ktx/issues/280)) ([7b00234](7b0023471e))
* align introduction subtitle width with page content ([#275](https://github.com/Kaelio/ktx/issues/275)) ([e5425b5](e5425b51a3))
* consolidate AI Resources into a single page ([#274](https://github.com/Kaelio/ktx/issues/274)) ([8050b59](8050b59f6e))
* document upgrading to the latest ktx version ([#273](https://github.com/Kaelio/ktx/issues/273)) ([7ece0b6](7ece0b63d3))
* remove product switcher from docs nav ([#272](https://github.com/Kaelio/ktx/issues/272)) ([07bbdef](07bbdefa14))

### Other Changes

* refresh star history chart [skip ci] ([bd3a375](bd3a375081))
* refresh star history chart [skip ci] ([50dec7b](50dec7bf64))
2026-06-09 14:41:43 +00:00
Andrey Avtomonov
7b0023471e
docs(agents): sync Opinionated Product Defaults guidance into AGENTS.md (#280) 2026-06-09 16:31:00 +02:00
Andrey Avtomonov
470802e58e
fix(cli): classify ktx setup abandonment as aborted, not a blank error (#278)
* fix(cli): classify ktx setup abandonment as aborted, not a blank error

ktx setup returned a non-zero exit code without throwing when a user
abandoned the interactive wizard, so the command telemetry recorded
outcome=error with no errorClass/errorDetail — an unactionable blank in
the errors dashboard, where most ktx setup "errors" were really people
backing out of the wizard.

Add annotateCommandOutcome() to the command span so the setup flow (the
decision-maker) records the true outcome: genuine step failures and
--no-input missing input become outcome=error with a self-diagnosing
reason, while interactive abandonment and project cancellation become
outcome=aborted and drop out of the error view.

Unify the exit code and telemetry through setupTerminalOutcome() so they
can never diverge: aborts now exit 0 (matching the entry-menu Exit,
project cancel, and a confirmed Ctrl+C), while failures and automation
errors still exit 1.

* fix(cli): treat non-TTY setup missing-input as an error, not an abort

setupTerminalOutcome classified `missing-input` by `args.inputMode`, but
`auto` only means "interactive if a TTY is attached". A piped/CI `ktx
setup` without `--no-input` and without `--yes` is still `auto`, yet the
project and agents steps return `missing-input` there without ever
prompting (e.g. "pass --yes to create a project outside an interactive
terminal"). Classifying that as `aborted` made a broken automation run
exit 0 — a silent failure.

Key the classification off actual interactivity instead: input enabled
AND `io.stdout.isTTY === true`. Non-interactive missing-input now exits
1 with a `KtxSetupMissingInput` reason; only a genuine interactive abort
exits 0. Adds a non-TTY regression test and fixes the abandonment test
to use a real TTY.
2026-06-09 12:53:15 +02:00
Andrey Avtomonov
66517fc320
feat(cli): add Slack community CTA on errors, crashes, setup, and help (#277)
* feat(cli): show Slack CTA on help and unexpected errors

* feat(cli): show Slack CTA after crashes

* feat(setup): show Slack community note after setup

* chore: refresh Python lockfile versions
2026-06-09 12:22:56 +02:00
Andrey Avtomonov
6b2f7c3365
fix(cli): ensure git committer identity during ktx setup (#276)
* fix(cli): ensure git committer identity during ktx setup

ktx setup threw "Failed to initialize git repository" when the project
directory was already a git repo with no commits and the machine had no
configured git identity (e.g. a fresh Mac with no ~/.gitconfig).
GitService only set the identity on the path where it created the repo
itself, so the bootstrap commit had no resolvable committer.

Carry ktx's identity via GIT_AUTHOR/GIT_COMMITTER env on the shared
git client so every commit succeeds regardless of whether ktx created
the repo, without mutating the user's repo config. Also preserve the
underlying git error when rethrowing so the failure is diagnosable in
telemetry and actionable for the user.

* chore: sync uv.lock ktx-daemon and ktx-sl versions to 0.10.0
2026-06-09 12:10:02 +02:00
github-actions[bot]
bd3a375081 chore: refresh star history chart [skip ci] 2026-06-09 07:21:47 +00:00
Luca Martial
e5425b51a3
docs: align introduction subtitle width with page content (#275)
The introduction page is special-cased as a hero in the docs route, so it
hand-rolls its own title and subtitle instead of using Fumadocs'
DocsTitle/DocsDescription. The subtitle was capped at max-w-2xl (672px),
wrapping ~230px narrower than the heading and body, which read as
misaligned compared to every other docs page. Match the heading's
max-w-full so the subtitle fills the same content column.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 00:49:38 -04:00
Luca Martial
8050b59f6e
docs: consolidate AI Resources into a single page (#274)
* docs: consolidate AI Resources into a single page

The AI Resources section was four pages (agent-quickstart, markdown-access,
agent-instructions, prompt-recipes) that repeated the same docs-consumption
guidance. Collapse them into one page at /docs/ai-resources covering markdown
endpoints, retrieval order, the task router, agent instructions, prompts, and
guardrails.

Also fix a stale claim: the page actions are a single "Copy as Markdown"
button, not the documented "Copy MD / View MD / Copy MDX" trio.

Update the cross-references in README, the introduction cards, the quickstart,
and the llms.txt entry points to the consolidated page.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(docs-site): redirect retired AI Resources slugs, preserving .md route

Redirect the retired per-page slugs (/docs/ai-resources/*) to the consolidated
page. Because Next evaluates redirects before the .md rewrite, a single
catch-all would 308 a cached per-page Markdown URL to the HTML page and break
the agent Markdown contract. Match the .md variant first and keep its suffix so
it lands on /docs/ai-resources.md.

Extend the routing test to assert both the HTML and .md redirects, and that
following the .md URL end to end serves text/markdown.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs: move AI Resources under the Community & Resources section

As a single page, AI Resources rendered as an orphaned, unbolded link wedged
between the top-level multi-page sections instead of as a section of its own.
Move it under Community (renamed "Community & Resources") so it renders as a
normal child link, consistent with how the single-page Configuration section
already works.

Redirect the former top-level URL and the retired per-page slugs (HTML and .md,
the .md variants first so cached Markdown URLs keep their suffix) to the new
home, relabel the llms.txt group to match, and repoint the README, introduction,
quickstart, and llms.txt cross-links.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 00:28:56 -04:00
Luca Martial
7ece0b63d3
docs: document upgrading to the latest ktx version (#273)
* docs: document upgrading to the latest ktx version

Install instructions existed but nothing told users how to upgrade an
existing install. Add a minimal upgrade note to the README and to the
quickstart install section showing `npm install -g @kaelio/ktx@latest`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* nitpick

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:10:36 -04:00
Luca Martial
07bbdefa14
docs: remove product switcher from docs nav (#272)
The docs site nav carried a "Products" dropdown listing both ktx and
the legacy Kaelio agent platform. On the ktx docs, a co-equal product
switcher framed ktx as one of two products and gave the unrelated
legacy product equal billing. Remove it so the ktx docs stay focused;
cross-product discovery belongs at the docs.kaelio.com apex, not in the
ktx nav.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 19:53:29 -04:00
github-actions[bot]
50dec7bf64 chore: refresh star history chart [skip ci] 2026-06-08 18:55:50 +00:00
semantic-release-bot
48676c74fa chore(release): 0.10.0 [skip ci]
## [0.10.0](https://github.com/Kaelio/ktx/compare/v0.9.0...v0.10.0) (2026-06-08)

### Features

* add GitHub star nudges to CLI build view and docs sidebar ([#271](https://github.com/Kaelio/ktx/issues/271)) ([795a974](795a97485a))
* **cli:** add channel-aware update notifier ([#265](https://github.com/Kaelio/ktx/issues/265)) ([698efdc](698efdcef8))
* **cli:** add ingest LLM rate-limit governor with paced retries ([#261](https://github.com/Kaelio/ktx/issues/261)) ([c3d8ced](c3d8cedb0b))
* **mysql:** implement columnStats using INFORMATION_SCHEMA.STATISTICS ([#233](https://github.com/Kaelio/ktx/issues/233)) ([18245c2](18245c2373))
* **setup:** apply per-role LLM model presets, remove --llm-model ([#268](https://github.com/Kaelio/ktx/issues/268)) ([2c18a62](2c18a62de4))
* **setup:** wizard prompt tweaks and quieter query-history filter output ([#259](https://github.com/Kaelio/ktx/issues/259)) ([c2beaf7](c2beaf7d55))
* **telemetry:** collect PostHog $exception error reports in CLI and daemon ([#262](https://github.com/Kaelio/ktx/issues/262)) ([fb7b94b](fb7b94b60e))

### Bug Fixes

* **docs-site:** stop doubling the /ktx basePath on alias-host redirects ([#263](https://github.com/Kaelio/ktx/issues/263)) ([d3e20df](d3e20df1d5))
* **ingest:** drive work-unit progress from tool calls, not turn counts ([#269](https://github.com/Kaelio/ktx/issues/269)) ([2896f9f](2896f9fb91))
* **sl:** stop baking drift-prone counts into overlay summaries ([#270](https://github.com/Kaelio/ktx/issues/270)) ([5232578](5232578d44))
* **telemetry:** preserve driver error class and code in connection_test ([#260](https://github.com/Kaelio/ktx/issues/260)) ([ec7edf8](ec7edf8f50))

### Documentation

* add serving-phase diagram to the introduction page ([#264](https://github.com/Kaelio/ktx/issues/264)) ([377f21a](377f21acd7))
* minor README and docs-site touch-ups ([#266](https://github.com/Kaelio/ktx/issues/266)) ([bf1fe97](bf1fe9748e))
* **site:** add Products dropdown to ktx docs navbar ([#267](https://github.com/Kaelio/ktx/issues/267)) ([dc39eb7](dc39eb7ef9))

### Other Changes

* refresh star history chart [skip ci] ([0d0ea55](0d0ea55184))
* refresh star history chart [skip ci] ([2914407](2914407f09))
* refresh star history chart [skip ci] ([d142274](d14227468b))
* refresh star history chart [skip ci] ([5a88210](5a8821073b))
* refresh star history chart [skip ci] ([8eb1cd3](8eb1cd3e79))
2026-06-08 14:47:15 +00:00
Andrey Avtomonov
795a97485a
feat: add GitHub star nudges to CLI build view and docs sidebar (#271)
* feat: load star count during context builds

* docs: document star prompt opt-out

* fix: initialize demo context star count

* feat(docs-site): add GitHub star count widget to docs sidebar

* test: isolate star-prompt build-view tests from ambient CI env
2026-06-08 16:14:56 +02:00
Andrey Avtomonov
5232578d44
fix(sl): stop baking drift-prone counts into overlay summaries (#270)
The auto-generated semantic-layer overlay description embedded
measure/segment/column counts that were computed once and never
recomputed, so the summary drifted and misreported its source after
measures were later appended. Make the auto fallback count-free, since
those counts are already rendered live from the body at `ktx sl list`/
`read` time; this removes the drift class without ever overwriting
human-authored descriptions (the fill-only-when-empty guard is untouched).

Adds a regression test that fails on main and passes after the fix, plus
guards for description preservation and the no-measures fallback.
2026-06-08 15:58:12 +02:00
Andrey Avtomonov
2c18a62de4
feat(setup): apply per-role LLM model presets, remove --llm-model (#268)
* feat(setup): write per-role llm model presets

* feat(setup): remove llm model setup flag

* chore(setup): update llm preset guidance

* docs(setup): document llm model presets

* chore(release): sync uv.lock to 0.9.0

* fix(cli): make sl query --execute work on secret-backed connections

sl query --execute used a parallel SQL executor (createDefaultLocalQueryExecutor)
that passed connection.url verbatim into pg, so file:/env: secret references
failed with "SASL: SCRAM-SERVER-FIRST-MESSAGE: client password must be a string".

Collapse onto the connector-based executor already used by MCP and ingest
(createKtxCliIngestQueryExecutor), which resolves secret references and supports
every driver. Delete the now-dead local/postgres/sqlite query executors, their
tests, and the orphaned hasLocalQueryExecutor driver flag.

* docs(agents): require one implementation per capability

Add a design-reasoning default and a matching self-check question telling agents
to route callers through a single shared implementation of a capability rather
than forking a parallel one, and to fix the shared layer rather than patch one
branch. Encodes the lesson from a divergent SQL-execution-path bug, stated
generally.

CLAUDE.md is a symlink to AGENTS.md, so both agent-instruction files are covered.
2026-06-08 15:30:48 +02:00
Andrey Avtomonov
2896f9fb91
fix(ingest): drive work-unit progress from tool calls, not turn counts (#269)
The ingest HUD showed "step 70/40" because the Claude subscription runtime
re-derived a per-turn counter that could not match the SDK's num_turns and
overshot the maxTurns budget. Replace the turn-based work_unit_step heartbeat
with a real, observed tool-call count (no denominator), report
metrics.stepCount from the SDK's authoritative num_turns, and delete the
brittle countsAsAssistantTurn denylist plus the now-unused onStepFinish
callback across the runtime port and all three runtimes. Reconcile and curator
progress move to the same tool-call heartbeat.
2026-06-08 15:30:35 +02:00
Mayorkun Ayanshina
18245c2373
feat(mysql): implement columnStats using INFORMATION_SCHEMA.STATISTICS (#233)
* feat(mysql): implement columnStats using INFORMATION_SCHEMA.STATISTICS

Enable column cardinality statistics for the MySQL connector by querying
INFORMATION_SCHEMA.STATISTICS, which provides index-based cardinality
estimates without requiring additional permissions.

- Add generateColumnStatisticsQuery() to KtxMysqlDialect
- Add getColumnStatistics() and columnStats() to KtxMysqlScanConnector
- Flip columnStats capability from false to true
- Add MysqlStatsRow and KtxMysqlColumnStatisticsResult interfaces
- Add tests for dialect query generation and connector stats retrieval
- Update dialect conformance fixture for mysql

* fix(mysql): filter to leading index columns to avoid inflated cardinality

Add AND SEQ_IN_INDEX = 1 to INFORMATION_SCHEMA.STATISTICS query to
ensure only leading index columns are returned. For composite indexes,
non-leading columns report the cardinality of the index prefix rather
than the column's own distinct count, which inflates distinctCount.

Add regression test asserting SEQ_IN_INDEX = 1 is present in the query.

* fix: add trailing newline to dialect.test.ts

---------

Co-authored-by: Andrey Avtomonov <andreybavt@gmail.com>
2026-06-08 12:21:19 +02:00
github-actions[bot]
0d0ea55184 chore: refresh star history chart [skip ci] 2026-06-08 07:52:04 +00:00
Andrey Avtomonov
dc39eb7ef9
docs(site): add Products dropdown to ktx docs navbar (#267)
Surface ktx (primary) and Kaelio Platform docs via a navbar product
menu so users can switch between the two docs sites; ktx stays first.
2026-06-07 23:15:21 +02:00
github-actions[bot]
2914407f09 chore: refresh star history chart [skip ci] 2026-06-07 07:30:35 +00:00
Luca Martial
bf1fe9748e
docs: minor README and docs-site touch-ups (#266)
- Link the Y Combinator badge and the docs "by Kaelio" label
- Add a maintainer line to the README
- Set the npm author field on @kaelio/ktx

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 22:32:08 -04:00
Andrey Avtomonov
698efdcef8
feat(cli): add channel-aware update notifier (#265)
* feat(cli): show cached update notices after commands

* docs(cli): describe update notices

* fix(cli): type update check environment

* fix(cli): decouple update notice display from refresh and harden suppression

Display a cached "update available" notice based solely on the lastNoticeAt
24h throttle, independent of checkedAt refresh freshness, matching the design's
independent display/refresh decisions. Suppress the check unconditionally under
--json, CI, and non-TTY before consulting output-mode preferences, so a
KTX_OUTPUT=pretty override can no longer make CI/non-TTY contexts phone npm.
2026-06-06 10:42:10 +02:00
Luca Martial
377f21acd7
docs: add serving-phase diagram to the introduction page (#264)
* feat(docs): add serving-phase diagram to the introduction page

The introduction's "How ktx works" section described both the ingest and serve sides but only rendered the ingestion diagram. Add a live, theme-aware React Flow diagram for the serving phase (agent <-> ktx via MCP -> context layer + database) so both phases are shown, with a matching content test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(diagram-studio): relabel context edge and use right-angle routing

The hub->context edge searches and reads definitions, not just searches; relabel it "search + read". Route the serving search/read-only edges with smoothstep (right angles) to match the docs diagram. (The README PNG is a baked export and is unchanged until re-exported from the studio.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(docs): point product-mechanics assertions at the FlowCanvas wrapper

product-mechanics renders via the shared FlowCanvas wrapper, so the ReactFlow config (nodesDraggable, zoomOnScroll, etc.) lives there now. Update the stale assertions that still expected those literals inline, fixing a pre-existing test failure.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(serving-diagram): shrink the boxes and drop OpenCode from the agent list

Reduce node dimensions, font sizes, padding, and the canvas height so the serving diagram renders ~25% smaller and more compact. Remove OpenCode from the agent's listed clients.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 19:22:45 -04:00
Luca Martial
d3e20df1d5
fix(docs-site): stop doubling the /ktx basePath on alias-host redirects (#263)
ktx.sh/ and docs.ktx.sh/ redirected to
https://docs.kaelio.com/ktx/ktx/docs/... (note the doubled /ktx) and 404'd.

The host-agnostic `source: "/"` redirect ran before the alias-host
canonicalizers, so it injected the /ktx basePath into the path on the alias
domains, which the alias catch-all then prepended a second time.

Reorder redirects() so alias-host canonicalization runs first, leaving the
generic root/docs rules for the local/canonical host only. The /stars
exclusion stays because redirects run before beforeFiles rewrites.

Add Host-spoofing regression tests (the prior tests only used localhost,
which never exercised the alias-host rules) and remove the vestigial
website/vercel.json, which the live ktx.sh routing already bypasses.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 15:05:22 -04:00
github-actions[bot]
d14227468b chore: refresh star history chart [skip ci] 2026-06-05 18:44:32 +00:00
Andrey Avtomonov
fb7b94b60e
feat(telemetry): collect PostHog $exception error reports in CLI and daemon (#262)
* feat(telemetry): add node exception reporter

* feat(telemetry): report node cli exceptions

* feat(telemetry): add daemon exception reporter

* feat(telemetry): report daemon exceptions

* docs(telemetry): document error reports

* fix(telemetry): pass redaction snapshots from node call sites

* test(telemetry): verify prepared node exception payload

* fix(telemetry): close daemon exception lifecycle gaps

* test(telemetry): verify prepared daemon exception payload

* test(telemetry): close error collection acceptance gaps

* test(telemetry): close posthog exception acceptance gaps
2026-06-05 19:36:21 +02:00
Andrey Avtomonov
c3d8cedb0b
feat(cli): add ingest LLM rate-limit governor with paced retries (#261)
* feat(cli): add ingest rate limit governor

* feat(cli): wire ingest rate-limit config

* feat(cli): report provider rate-limit signals

* feat(cli): show ingest rate-limit waits

* fix(cli): complete rate-limit event coverage

* fix(cli): abort ingest provider calls cleanly

* fix(cli): propagate ingest cancellation

* fix(cli): reject pre-aborted ingest rate-limit waits

* fix(cli): honor Claude rate-limit reset waits

* fix(cli): retry thrown Codex rate-limit failures

* fix(cli): type Claude rate-limit result details

* fix(cli): emit ingest rate-limit countdowns from rejected signals

* fix(cli): report ai sdk rate-limit header utilization

* fix(cli): gate LLM rate-limit retries on the governor budget

The AI SDK and Codex runtimes retried 429 / opaque rate-limit failures up
to 6-7 times with no backoff when constructed without a RateLimitGovernor
(scan, memory, setup) or with pacing disabled, ignoring Retry-After and
worsening the limit. The outer retry loop only cooperates with the
governor's pause, so without active pacing there is no backoff to apply.

Route the retry bound through a single source: RateLimitGovernor
.maxRetryAttempts(), which returns retry.maxAttempts when enabled and 1
(no outer retry) when absent or disabled. All three runtimes (ai-sdk,
codex, claude-code) now use it, so ingest.rateLimit.retry.maxAttempts
genuinely controls attempts and the hard-coded 6 (plus Codex's off-by-one
extra attempt) is gone. Backend-native retry (e.g. the AI SDK's maxRetries)
still handles transient 429s.

Also correct the ktx.yaml docs for maxWaitMs (caps each wait, not the whole
run) and maxAttempts, and sync uv.lock ktx-sl/ktx-daemon to 0.9.0.
2026-06-05 12:10:27 +02:00
github-actions[bot]
5a8821073b chore: refresh star history chart [skip ci] 2026-06-04 18:53:21 +00:00
Andrey Avtomonov
ec7edf8f50
fix(telemetry): preserve driver error class and code in connection_test (#260)
Native connector test failures were flattened to `new Error(message)`,
collapsing every driver's error class to `Error` and dropping `.code` /
`.number`. connection_test telemetry could therefore not tell a SQL Server
login rejection (ELOGIN / 18456) apart from a network or TLS error, and the
only field that varied was a raw message.

Connectors now return `connectorTestFailure(error)`, which preserves the
original driver error as `cause`, and `testNativeConnection` re-throws that
cause. `scrubErrorClass` then records the real class (e.g. ConnectionError)
and `formatErrorDetail` keeps the code prefix (e.g. "ELOGIN: ..."). The
helper is the single source of truth for the failure shape across all seven
native connectors. User-facing terminal output is unchanged.
2026-06-04 14:51:14 +02:00
Andrey Avtomonov
c2beaf7d55
feat(setup): wizard prompt tweaks and quieter query-history filter output (#259)
Setup wizard flow tweaks:
- Add a reveal-tail password prompt (reveal-password-prompt.ts) that unmasks
  the last few characters of a typed/pasted secret, and wire it into the setup
  prompt adapter in place of clack's password(); adds the @clack/core dep.
- Reorder wizard select options: surface "Paste a key" before the
  environment-variable option across embeddings/models/sources, promote
  Metabase/Notion in the source list, put Git URL before Local path, reorder
  the Notion crawl-mode choices, and relabel the sources "Done" action.

Query-history filter picker output:
- Collapse the per-template parse-failure lines into a single count in the
  setup output and route the full template-id list to --debug stderr.
- Model parse failures as a structured parseFailedTemplateIds field instead of
  warning strings.
- Add a privacy-safe query_history_filter_completed telemetry event
  (counts/enums only), mirrored into the Python daemon schema.
2026-06-04 14:11:08 +02:00
github-actions[bot]
8eb1cd3e79 chore: refresh star history chart [skip ci] 2026-06-04 07:45:37 +00:00
semantic-release-bot
7ba948a135 chore(release): 0.9.0 [skip ci]
## [0.9.0](https://github.com/Kaelio/ktx/compare/v0.8.0...v0.9.0) (2026-06-03)

### Features

* add codex llm backend for ktx runtime work ([#253](https://github.com/Kaelio/ktx/issues/253)) ([494618a](494618ab14))
* **cli:** consistent connection setup recovery and build-time gate ([#257](https://github.com/Kaelio/ktx/issues/257)) ([ce1516b](ce1516b357))
* **cli:** guide next action at end of ktx setup, not reruns ([#256](https://github.com/Kaelio/ktx/issues/256)) ([45aa95d](45aa95d2cc))
* **cli:** stream plain ktx ingest progress to stderr (KLO-726) ([#251](https://github.com/Kaelio/ktx/issues/251)) ([13774bf](13774bfcef))
* **query-history:** scope mining to modeled schemas by default ([#258](https://github.com/Kaelio/ktx/issues/258)) ([e70ae1e](e70ae1e63b))
* **telemetry:** include error details for failures ([#254](https://github.com/Kaelio/ktx/issues/254)) ([6da8c34](6da8c3452a))

### Bug Fixes

* **ingest:** recover textual-conflict gate failures; fix query-history adapter ([#255](https://github.com/Kaelio/ktx/issues/255)) ([f5dea9a](f5dea9a089))

### Other Changes

* refresh star history chart [skip ci] ([9d3a0b7](9d3a0b751d))
* refresh star history chart [skip ci] ([74c6076](74c6076b72))
* refresh star history chart [skip ci] ([d01abe6](d01abe6f3c))
* revert repo references to Kaelio/ktx and remove rename-resilience ([#252](https://github.com/Kaelio/ktx/issues/252)) ([41e20c9](41e20c9ce7)), closes [#250](https://github.com/Kaelio/ktx/issues/250) [#250](https://github.com/Kaelio/ktx/issues/250)
2026-06-03 21:50:59 +00:00
Andrey Avtomonov
e70ae1e63b
feat(query-history): scope mining to modeled schemas by default (#258)
* feat(query-history): structure SQL analysis table refs

* feat(query-history): qualify SQL analysis table refs

* feat(query-history): wire modeled scope floor through ingest

* chore(query-history): verify scope floor

* test(query-history): align daemon SQL batch endpoint contract

* feat(query-history): build scope from same-run scan catalog

* feat(query-history): fail open on scope-floor catalog failures

* chore(query-history): verify scope-floor v1 closure

* refactor(query-history): share scope membership

* feat(setup): apply derived query history filters

* docs: document derived query history filters

* fix(query-history): redact filter picker LLM prompt SQL

* fix(setup): run filter picker SQL analysis through managed daemon

* chore(query-history): verify filter picker v1 closure

* fix(query-history): fail open on partial service-account attribution

* fix(query-history): aggregate BigQuery users by execution count

* fix(query-history): aggregate Snowflake users by execution count

* fix(query-history): use BigQuery query info hash
2026-06-03 17:19:42 +02:00
Andrey Avtomonov
ce1516b357
feat(cli): consistent connection setup recovery and build-time gate (#257)
* feat(cli): block context build when a required connection fails its live test

A context build can take several minutes, so a connection that is
unreachable or misconfigured should stop the build up front instead of
failing partway through. Before the build starts, run a live connection
test for every primary- and context-source connection the build depends
on.

Each test's output is captured in a discarded buffer so raw error text
(and database paths) never reach the user; failures are surfaced only by
connection id and connector type, with a pointer to `ktx connection test
<id>` for the underlying error.

- Interactive setup lets the user fix the connection and retry without
  restarting, re-resolving targets so an added/removed/reconfigured
  connection is honored.
- `--no-input` exits non-zero and writes a failed context state with a
  failureReason, so scripts stop early and setup never reads as ready.

Extract the buffered command IO helper out of setup-databases into
src/io/buffered-command-io.ts so both call sites share one implementation.

* feat(cli): use recovery primitive for database setup

* feat(cli): use recovery primitive for source setup

* docs: document setup connection recovery

* fix(cli): close database recovery gaps

* fix(cli): target failing project in gate hint and preserve missing-input

Address two review findings on the connection-recovery work:

- The connection-gate failure hint emitted `ktx connection test <id>` with no
  --project-dir, so a setup run started with `--project-dir ./analytics` pointed
  users at cwd/KTX_PROJECT_DIR instead of the project that just failed. Emit the
  resolved project dir, matching the contextBuildCommands convention.

- The non-interactive database configure path returned `cancelled`, which the
  recovery primitive collapses to `failed`. Sibling paths still report
  `missing-input` for absent flags, so incomplete-flag runs were
  indistinguishable from real connection failures. The database wrapper now
  tracks the configure missing-input signal and restores the `missing-input`
  step status; the shared primitive keeps its four outcomes.
2026-06-03 11:08:46 +00:00
Andrey Avtomonov
f5dea9a089
fix(ingest): recover textual-conflict gate failures; fix query-history adapter (#255)
* fix(ingest): recover textual-conflict gate failures; fix query-history adapter

Two latent gaps in the isolated-diff local-ingest pipeline that can abort an
otherwise-successful ingest:

- Metabase: when a work-unit patch hit both a textual conflict and a post-merge
  dangling sl_ref, the after-textual-resolution branch returned a hard
  semantic_conflict and rolled back the whole job. It now runs the same
  repairGateFailure recovery the clean-apply branch already uses (re-validate,
  then commit the union of resolved + repaired paths), reaching parity.

- Query history: the historic-sql adapter was registered only when ktx.yaml had
  context.queryHistory.enabled=true, so `--query-history` threw "Adapter not
  available for local ingest". Registration now resolves the dialect from driver
  capability, since the explicit --query-history request is itself the opt-in;
  the config-gated helper is unchanged for status/setup/probes.

Adds the previously-missing tests for both paths.

* chore: sync uv.lock to 0.8.0 (regenerated with pinned uv 0.11.11)

* fix(ingest): drop ktx's own scan probes and dedup tables in query history

Query history (historic-sql) mined two kinds of noise back into context:

- ktx's own warehouse scan emits relationship- and column-profiling probes
  (the relationship_profile_values aggregation and the child_values/parent_values
  FK-overlap CTEs) into pg_stat_statements. shouldDropBySql now filters these
  ktx-owned, dialect-stable signatures so ktx introspection is not ingested as
  usage history.

- The same physical table appears both bare (accounts, via search_path) and
  schema-qualified (orbit_raw.accounts), producing duplicate per-table work
  units. canonicalizeTableIdentifiers collapses a bare name into its unique
  qualified form before work-unit keying; ambiguous names are left untouched.

On the orbit demo this removes ~35% of sampled query templates (ktx self-probes)
and ~45 duplicate per-table work units.

* docs(agents): add Design Reasoning Defaults section
2026-06-03 13:05:59 +02:00
github-actions[bot]
9d3a0b751d chore: refresh star history chart [skip ci] 2026-06-03 07:50:39 +00:00
Andrey Avtomonov
45aa95d2cc
feat(cli): guide next action at end of ktx setup, not reruns (#256)
Re-running setup was the dominant action for installs that completed setup but never ingested. Classify completion (incomplete | needs-context | needs-agents | ready) and drive one obvious next action per state: route a config-complete project straight to the build, point unbuilt-context users at `ktx ingest` instead of re-running setup or dropping to a bare shell, and confirm readiness for fully-set-up projects rather than reopening the edit menu.
2026-06-03 01:00:21 +02:00
Andrey Avtomonov
cb6a67c2d7 Make telemetry reliable across interrupts and headless installs
Three reliability gaps surfaced while auditing why PostHog numbers were
untrustworthy:

1. Interrupted commands lost their events. capture() is fire-and-forget and the
   only flush guarantee lived in a finally block, which SIGINT/SIGTERM skip — so
   Ctrl-C'ing a long ingest or an MCP client killing 'ktx mcp stdio' dropped the
   command event and any queued events. Add SIGINT/SIGTERM handlers (real-process
   entry only; never under test/programmatic io) that mark the active command
   span aborted, emit it, drain the emitter, then exit. Idempotent with the
   normal finally path via the single-consume command span.

2. Headless-first installs were invisible. loadTelemetryIdentity refused to mint
   an installId unless stdout was a TTY, so a machine whose first run was an
   IDE-launched MCP server or a script emitted nothing, ever. Mint on first run
   regardless of surface (still honoring CI/DO_NOT_TRACK/KTX_TELEMETRY_DISABLED),
   writing the one-time notice to stderr — safe under the MCP stdio protocol,
   which reserves stdout. Drop the now-unused stdoutIsTTY option.

3. No guard against silent emit regressions (the 0.7.0 scan_completed blackout).
   Add tests: the shared executePublicIngestTarget chokepoint emits exactly one
   ingest_completed on success and on the preflight-failure branch, and a
   database target invokes the scan that emits scan_completed; plus coverage for
   the aborted-flush helper.

Identity is unchanged otherwise: every event still attributes to the installId
in ~/.ktx/telemetry.json. No event/field changes, so Node<->Python schema parity
is untouched. Docs updated to reflect first-run-on-any-surface activation.
2026-06-02 23:19:37 +02:00
Andrey Avtomonov
2334a4b6e3 Emit ingest_completed once per target on every ingest path
emitIngestCompleted was called only in runKtxPublicIngest's plain/json loop,
so the foreground 'ktx ingest' view and all of 'ktx setup' — which delegate to
runContextBuild -> executePublicIngestTarget — never emitted the event. That
left ingest_completed near-useless for measuring ingestion.

Move the emit into executePublicIngestTarget, the single per-target chokepoint
every entrypoint funnels through: a thin wrapper now captures timing, runs the
existing steps (extracted to runIngestTargetSteps), and emits exactly once. The
telemetry echo targets deps.runtimeIo (the real user stream) so a capture
buffer used for step output doesn't swallow it. Thread project through the
context-build call site. No schema/field changes, so Node<->Python telemetry
parity is unaffected.

Add tests: the shared chokepoint emits exactly one ingest_completed for any
caller, and a multi-target run emits one per target with no double-emit.
2026-06-02 20:03:27 +02:00
Andrey Avtomonov
6da8c3452a
feat(telemetry): include error details for failures (#254) 2026-06-02 17:23:51 +02:00
Andrey Avtomonov
494618ab14
feat: add codex llm backend for ktx runtime work (#253)
* feat: add codex sdk runner foundation

* feat: parse codex runtime events

* feat: expose codex runtime mcp tools

* feat: add codex llm runtime

* feat: wire codex llm backend

* test: avoid Array.fromAsync in codex runner test

* docs: document codex llm backend

* fix: tighten codex runtime config ownership

* fix: use codex sdk env and thread options

* fix: parse codex sdk event shapes

* test: add codex backend live smoke

* docs: clarify codex backend isolation

* fix: drive codex loop metrics from mcp events

* fix: enforce codex local step budget

* docs: disclose codex isolation limits

* fix: count all codex agent steps and stream step callbacks live

The agent-loop step budget only counted completed mcp_tool_call items, so
built-in command_execution steps (which the public Codex SDK/CLI surface can
still expose) never decremented the budget, letting ingest/reconciliation run
past stepBudget until Codex stopped on its own. onStepFinish was also replayed
only after the whole stream drained, so live work_unit_step / reconciliation
progress appeared stuck until the Codex process exited.

collectEvents is now the single live step accumulator: it counts every
completed agent-action item via a shared isCompletedAgentStep predicate
(command_execution, mcp_tool_call, file_change, web_search), fires onStepFinish
as each step completes, and enforces the budget on that broader count. A
no-tool turn still counts as one step. toolFailures stays MCP-specific, since a
non-zero command exit is normal agent exploration, not a loop failure.

* test: align ingest llm-guard assertions with codex backend

The skip-llm ingest guard message now lists codex as a valid backend and
mentions a Claude Code/Codex session plus a codex setup hint, but this slow
suite test still asserted the pre-codex wording. Update it to match the
production message (already covered by the local-bundle-runtime unit test) and
add the codex setup-line assertion.

* fix: treat codex error:null tool calls as success

The Codex SDK serializes error: null on successful mcp_tool_call items, so
the failure check (item.error !== undefined) flagged every successful tool
call as failed with the empty-payload default "Codex turn failed". This
killed every ingest work unit under the codex backend before it could
produce a patch.

Key on status === 'failed' (authoritative, always set) and only treat a
populated error object as a failure. Add a regression test built from a
verbatim real-SDK event capture.

* fix: default codex backend to gpt-5.5 and report real probe errors

The previous default gpt-5.3-codex is an API-key-only model that the OpenAI
API rejects under ChatGPT-account (subscription) auth, so codex status/setup
failed with a misleading "authentication is not usable" message even though
auth was fine.

- Default codex model is now gpt-5.5 (works on both subscription and API-key
  auth); the curated setup picker offers gpt-5.5 / gpt-5.4 / gpt-5.4-mini and
  keeps free-form entry for account-specific ids (e.g. gpt-5.3-codex-spark).
- runCodexAuthProbe now distinguishes "model not available" from an auth
  failure and surfaces the real API error: collectEvents retains stream
  events when the SDK throws on a non-zero exit, and the API error JSON
  envelope is unwrapped to its human-readable message.
- The Codex isolation warning now renders inside the clack setup frame.
- Docs updated to gpt-5.5 with a note that *-codex ids require API-key auth.

* fix: require llm.models.default in status and match codex probe remediation

Status reported a project ready when a non-none LLM backend was configured
without llm.models.default, but the runtime (resolveModelSlots) hard-requires
it, so ingest/scan/memory threw after `ktx status` said the project was usable.
buildLlmStatus now fails for any non-none backend missing models.default and no
longer invents a fallback model for claude-code/codex.

Codex probe failures now carry a category-matched fix: a model-access failure
steers the user at llm.models.default instead of the auth/install remediation.
runCodexAuthProbe returns the fix and status consumes it; the message stays
self-sufficient so setup output is unchanged.

Docs: README now lists the codex backend and local Codex auth; ktx-setup.mdx
states --llm-model only accepts codex/default or gpt-*/codex-* ids.

Repaired four doctor fixtures that configured a backend without models.default
(the now-correctly-blocked config) and added coverage for the new behavior.
2026-06-02 13:57:11 +02:00
github-actions[bot]
74c6076b72 chore: refresh star history chart [skip ci] 2026-06-02 07:46:46 +00:00
Andrey Avtomonov
41e20c9ce7
chore: revert repo references to Kaelio/ktx and remove rename-resilience (#252)
The GitHub repo was renamed back from Kaelio/ktx-ai-data-agents-context to Kaelio/ktx, reverting the URL changes from #250 across package metadata, CI (codecov + star-history slugs), issue/security templates, the release runbook, and docs/install commands.

Also removes the rename-resilience machinery #250 added: semantic-release now reads the repository URL straight from package.json (Kaelio/ktx) again, so the repositoryUrl() derivation in scripts/semantic-release-config.cjs, its tests, and the rename note in docs/release.md are no longer needed.
2026-06-02 00:14:43 +02:00
Andrey Avtomonov
13774bfcef
feat(cli): stream plain ktx ingest progress to stderr (KLO-726) (#251)
* feat(cli): share public ingest progress adapter

* feat(cli): stream plain public ingest progress

* test(cli): update plain ingest progress assertions

* chore(cli): satisfy plain ingest progress checks

* fix(artifacts): expect plain ingest stderr progress in installed-CLI smoke

* ci(coverage): make Codecov upload non-fatal and fix repo slug

The Coverage job failed because the Codecov upload returned
'Repository not found' while fail_ci_if_error was true, turning a
Codecov-side issue into a hard CI failure even though all tests pass.

- Set fail_ci_if_error: false on both uploads so Codecov outages or an
  unlinked repo no longer break CI (upload stays best-effort).
- Correct the stale slug Kaelio/ktx -> Kaelio/ktx-ai-data-agents-context
  to match the actual GitHub repo (aligns with main).

* fix(cli): isolate query-history failure capture from scan output

The plain public-ingest progress path passes one captured IO as the
target-level `io`. With progress deps set, both the schema scan and the
query-history ingest resolved their capture to that same shared buffer,
so a non-actionable query-history failure surfaced leftover scan report
text (e.g. "Mode: enriched") as the skipped-facet detail instead of the
real query-history message.

Give the query-history ingest a phase-local capture while preserving the
flow-to-io branch the foreground context-build view relies on.

---------

Co-authored-by: Andrey Avtomonov <7889985+andreybavt@users.noreply.github.com>
2026-06-01 23:31:31 +02:00
github-actions[bot]
d01abe6f3c chore: refresh star history chart [skip ci] 2026-06-01 19:42:24 +00:00
semantic-release-bot
41cccc3448 chore(release): 0.8.0 [skip ci]
## [0.8.0](https://github.com/Kaelio/ktx-ai-data-agents-context/compare/v0.7.0...v0.8.0) (2026-06-01)

### ⚠ BREAKING CHANGES

* **cli:** remove fast mode; ktx ingest always builds enriched context (KLO-721) (#237)

### Features

* **cli:** profile ingest runs and split model vs tool time ([#249](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/249)) ([21744fc](21744fc520))
* **cli:** remove fast mode; ktx ingest always builds enriched context (KLO-721) ([#237](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/237)) ([3f0d11e](3f0d11e07d))
* **cli:** shell completion for commands, flags, and entity names ([#244](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/244)) ([d320d54](d320d54ab2)), closes [#243](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/243)
* README architecture diagrams + React Flow diagram studio ([#245](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/245)) ([ba5bb92](ba5bb92ab7))
* report MCP client telemetry ([#242](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/242)) ([2e5f7f2](2e5f7f25aa))
* **telemetry:** enable PostHog GeoIP enrichment ([#243](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/243)) ([95a2653](95a265323a))
* trim MCP query response payloads ([#240](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/240)) ([25f639f](25f639fba2))

### Bug Fixes

* **brand:** README lockup wordmark in Outfit to match docs-site ([#246](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/246)) ([1959f49](1959f493d6))
* **cli:** align Notion setup credential to --source-auth-token-ref ([#236](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/236)) ([637891f](637891f030))
* **cli:** treat artifact-producing ingests with failures as partial ([#238](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/238)) ([53a6f8d](53a6f8d111))
* **release:** point repository URLs at renamed GitHub repo ([#250](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/250)) ([41f5279](41f52797de))

### Documentation

* **ktx skill:** harden setup guidance from agent-driven demo run ([#247](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/247)) ([5faa16b](5faa16b32c))
* **readme:** add launch video to README hero ([#248](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/248)) ([22ddf55](22ddf5524c))

### Continuous Integration

* normalize star-history.svg trailing newline ([#241](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/241)) ([cbbcf8e](cbbcf8e8bd)), closes [#240](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/240)
* push star-history refresh to protected main with RELEASE_PAT ([#239](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/239)) ([ba06f70](ba06f7078a))
* refresh README star history chart twice daily ([08d08d8](08d08d8ea0))
* stop tombi reformatting uv.lock and sync lock to 0.7.0 ([#235](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/235)) ([8ebc4ce](8ebc4ce107))

### Other Changes

* refresh star history chart [skip ci] ([c196d1f](c196d1f192))
* refresh star history chart [skip ci] ([2058c26](2058c26e84))
* refresh star history chart [skip ci] ([54d6e87](54d6e87733))
* upgrade dependencies and tooling ([#232](https://github.com/Kaelio/ktx-ai-data-agents-context/issues/232)) ([d53cdac](d53cdac366))
2026-06-01 18:09:14 +00:00
Andrey Avtomonov
41f52797de
fix(release): point repository URLs at renamed GitHub repo (#250)
* fix(release): point repository URLs at renamed GitHub repo

The GitHub repo was renamed from Kaelio/ktx to
Kaelio/ktx-ai-data-agents-context. semantic-release reads repositoryUrl
from package.json's repository field and the @semantic-release/github
plugin failed verifyConditions with EMISMATCHGITHUBURL because it no
longer matched the live clone URL.

Update every Kaelio/ktx reference to the renamed repo: package metadata
(root + CLI repository/bugs/homepage), the codecov upload slugs and
star-history slug in CI, the issue-template and security-advisory links,
the release runbook, and all docs/install commands.

* fix(release): derive semantic-release repositoryUrl from the CI repo

@semantic-release/github exact-matches repositoryUrl against the live
GitHub clone_url (no redirect following), so any repo rename re-breaks the
release when repositoryUrl is the static package.json value.

Derive repositoryUrl from the runner's GITHUB_REPOSITORY/GITHUB_SERVER_URL
so it always tracks the current repo name. A future rename (including back
to Kaelio/ktx) now resolves with no code change. Outside CI the option is
omitted, so semantic-release falls back to package.json as documented.

The package.json repository field stays ktx-ai-data-agents-context as
npm-display metadata, decoupled from the release-time match.
2026-06-01 20:07:24 +02:00
Andrey Avtomonov
9133d243e8 Update demo warehouse URL 2026-06-01 16:44:41 +02:00
Andrey Avtomonov
21744fc520
feat(cli): profile ingest runs and split model vs tool time (#249)
* feat(cli): profile ingest runs to find where wall-clock time goes

Add opt-in profiling for `ktx ingest`. Each timed phase, work unit, and
agent loop now records durationMs / step count / token usage in the
trace, and a post-run aggregator rolls them up into a "where did the
time go" report printed to stderr.

Enable per run with KTX_PROFILE_INGEST (1/true -> human table, json ->
raw structured profile) or persistently via `ingest.profile` in
ktx.yaml. The json form emits raw milliseconds, token counts, and a
summary.headline one-line diagnosis so coding agents can parse it
directly; json wins when both env and config request profiling.

- runtime-port: RunLoopMetrics (totalMs, usage, stepCount,
  stepBoundariesMs) plus onMetrics callbacks on text/object generation
- ai-sdk + claude-code runtimes: capture per-loop timing and token usage
- work-unit-executor and stages 3/4: thread metrics into trace events
- ingest-bundle.runner: time worktree / triage / clustering / index /
  reconcile / squash phases and emit the profile in a finally block
  (best-effort; never affects the run outcome)
- ingest-profile: new trace+transcript aggregator with table/json formatters
- config: ingest.profile flag; docs: profiling section in ktx-ingest.mdx

* fix(cli): flush tool-call logs before reading ingest profile

Tool transcripts are appended fire-and-forget so the agent hot path never
blocks on logging. The ingest profiler read them before the writes settled,
so per-work-unit toolMs (and the model-vs-tool split derived from it) could
be incomplete. Track in-flight appends and expose flushToolCallLogs() —
bounded by a timeout so it can never hang — and flush before the profiler
reads the transcript.
2026-06-01 15:49:17 +02:00
Andrey Avtomonov
22ddf5524c
docs(readme): add launch video to README hero (#248)
Add a clickable launch-video poster (linking to YouTube) directly after
the intro note and before the architecture diagrams. GitHub Markdown can
not embed a YouTube player, so the poster image links out instead.
2026-06-01 13:42:42 +00:00
Andrey Avtomonov
5faa16b32c
docs(ktx skill): harden setup guidance from agent-driven demo run (#247)
Fold field-tested fixes into the ktx skill, verified against current CLI source:

- prefer file: secret refs over env: (env: re-resolves per-process and resolves
  empty in later ingest/mcp shells)
- pass --skip-agents on data-only setup runs; explain the trailing agent step's
  misleading exit 1 on otherwise-successful runs
- dbt ignores --source-warehouse-connection-id (maps by table name); required
  only for Metabase/Looker/LookML
- never go silent during slow setup/ingest: poll .ktx mtimes and post progress
  so a long run does not look stuck
- judge readiness from verdict, connections[].status, localStats.semanticLayer
  and wikiPages; perConnection under-reports
- add troubleshooting entries for the 'Run in a TTY' exit 1 and secrets that
  resolve empty only during ingest/mcp
2026-06-01 12:08:58 +00:00
Andrey Avtomonov
1959f493d6
fix(brand): README lockup wordmark in Outfit to match docs-site (#246) 2026-06-01 11:18:37 +00:00
Andrey Avtomonov
ba5bb92ab7
feat: README architecture diagrams + React Flow diagram studio (#245)
Replace the tall portrait README ingestion SVG with two landscape
diagrams — "1 · Ingestion" (build the context layer) and "2 · Serving"
(agents query it through MCP) — wired in as transparent 2x PNGs that
read on GitHub light and dark.

Add docs-site/diagram-studio: a static React Flow page with custom
themed nodes and the inlined ktx mascot that renders both diagrams and
exports them to PNG via html-to-image (the diagrams' reproducible
source). Remove the superseded ingestion-flow SVGs.
2026-06-01 12:06:27 +02:00
Andrey Avtomonov
d320d54ab2
feat(cli): shell completion for commands, flags, and entity names (#244)
* feat(completion): complete known argument values

* fix(completion): hide Commander-hidden subcommands from completions

Replace the `__`-prefix name heuristic with Commander's `_hidden` flag so
internal subcommands registered with { hidden: true } (e.g. `mcp serve-internal`)
are excluded from completions, mirroring `ktx --help`.

* test: cover wiki and sl read command routing

* test: cover raw wiki and sl reads

* feat: add wiki read command

* feat: add sl read command

* feat: complete read command entity names

* docs: document wiki and sl read commands

* test: include read commands in command tree

* feat(sl): read and validate unique sources by name

* feat(sl): make read and validate connection id optional

* fix(completion): dedupe semantic source names

* docs(sl): document connection-optional read and validate

* fix(sl): require connection id for query command

* docs(sl): clarify query connection requirement

* fix(completion): don't resolve option values as subcommands

resolveCommand skipped flag tokens but not the value consumed by a
value-taking option in the `--flag value` form, so a connection id like
`query` was matched as the `sl query` subcommand and yielded no `sl`
completions. Track value-taking options and skip their consumed value
before matching subcommands.

* test(telemetry): assert first-run notice via TELEMETRY_NOTICE constant

CI (which tests this branch merged with main) failed because #243 changed
the first-run notice wording in identity.ts (dropped "anonymous") but left
this test grepping for the old literal 'ktx collects anonymous usage data',
so indexOf returned -1. Assert against the exported TELEMETRY_NOTICE
constant instead so the test tracks the source of truth and cannot drift
when the notice text changes again.
2026-05-31 23:44:33 +02:00
github-actions[bot]
c196d1f192 chore: refresh star history chart [skip ci] 2026-05-31 18:29:55 +00:00
github-actions[bot]
2058c26e84 chore: refresh star history chart [skip ci] 2026-05-30 18:28:06 +00:00
Andrey Avtomonov
95a265323a
feat(telemetry): enable PostHog GeoIP enrichment (#243)
Set disableGeoip: false on the CLI telemetry client so events are enriched with approximate, IP-based location at ingest. Update the first-run notice, public telemetry docs, and the AGENTS telemetry policy to drop the prior "anonymous" wording to match.
2026-05-30 18:33:14 +02:00
Andrey Avtomonov
2e5f7f25aa
feat: report MCP client telemetry (#242) 2026-05-30 18:00:25 +02:00
Andrey Avtomonov
25f639fba2
feat: trim MCP query response payloads (#240) 2026-05-30 17:54:24 +02:00
Andrey Avtomonov
cbbcf8e8bd
ci: normalize star-history.svg trailing newline (#241)
The star-history refresh workflow committed the API's SVG verbatim, but the
response has no trailing newline. Because the refresh commit uses [skip ci],
the file never ran end-of-file-fixer at commit time, so pre-commit's
`--all-files` run failed end-of-file-fixer on every open PR (e.g. #240), even
PRs that never touched the file.

Normalize the downloaded SVG to exactly one trailing newline in the workflow
(idempotent, so the "unchanged" guard still works), and fix the currently
committed file so open PRs go green now.
2026-05-30 17:44:27 +02:00
github-actions[bot]
54d6e87733 chore: refresh star history chart [skip ci] 2026-05-30 14:02:55 +00:00
Andrey Avtomonov
ba06f7078a
ci: push star-history refresh to protected main with RELEASE_PAT (#239)
The scheduled star-history workflow checked out with the default
GITHUB_TOKEN, so its git push to main was rejected by the branch
protection hook (GH006). Check out with RELEASE_PAT instead, matching
release.yml, whose semantic-release step already pushes to the protected
main branch with the same token.
2026-05-30 16:01:47 +02:00
Andrey Avtomonov
08d08d8ea0 ci: refresh README star history chart twice daily
Point the README chart at a committed assets/star-history.svg instead of
the star-history API URL so GitHub serves it directly and bypasses the Camo
proxy cache. A scheduled workflow regenerates the SVG at 06:00/18:00 UTC,
busting star-history's server-side cache, and commits it when it changes.
2026-05-30 12:07:15 +02:00
Andrey Avtomonov
53a6f8d111
fix(cli): treat artifact-producing ingests with failures as partial (#238)
* fix(cli): derive ingest outcomes from saved artifacts

* fix(cli): treat artifact-producing ingests with failures as partial

* fix(cli): route memory-flow run status through shared ingest outcome

* fix(cli): treat partial ingest as saved context in setup status

* test(cli): align memory-flow replay expectations with partial ingests
2026-05-30 00:42:59 +02:00
Andrey Avtomonov
3f0d11e07d
feat(cli)!: remove fast mode; ktx ingest always builds enriched context (KLO-721) (#237)
Fast mode (the ktx ingest --fast/--deep database-ingest depth toggle) is removed.
ktx ingest now always builds the full enriched ("deep") context. There is no
structural fallback: a database connection without a configured model and
embeddings fails the enrichment-readiness preflight before any work runs, with
a 'Run ktx setup to configure a model and embeddings' hint.

- Remove --fast/--deep flags, the per-connection context.depth field, and the
  ktx setup depth prompt (delete setup-database-context-depth.ts).
- Rename ingest-depth.ts -> connection-drivers.ts; ingest always requests scan
  mode 'enriched'; readiness gate (enrichmentReadinessGaps) runs for every
  database target.
- Drop the database-context-depth telemetry step (Node + Python schema mirrors
  regenerated).
- Update CLI, setup, context-build view, docs, the public ktx skill, and the
  release-smoke / artifacts scripts (now assert the no-LLM guard failure).

ktx status --fast (a separate network-probe flag) is unchanged.

Follow-ups: KLO-726 (live progress for ktx ingest --all), KLO-727 (restore
credentialed successful-ingest release smoke coverage).
2026-05-29 17:41:04 +02:00
Andrey Avtomonov
637891f030
fix(cli): align Notion setup credential to --source-auth-token-ref (#236)
Notion's setup path read --source-api-key-ref while writing the auth_token_ref
config field, so --source-auth-token-ref was silently dropped. Align Notion to
the flag=field convention every other connector follows: it now reads
--source-auth-token-ref, and --source-api-key-ref becomes Metabase-only.

Also add validation rejecting any credential-ref flag not applicable to the
chosen --source, with a pointer to the correct flag, closing the silent-drop
class for all connectors.

Update CLI-reference docs, the ktx skill Notion example, and tests.

Fixes KLO-724.
2026-05-29 17:23:46 +02:00
Andrey Avtomonov
8ebc4ce107
ci: stop tombi reformatting uv.lock and sync lock to 0.7.0 (#235)
The pre-commit job failed because tombi-format reformats uv.lock to a
layout uv does not produce, so once CI's uv sync re-resolved the stale
lock (workspace members still at 0.6.0) and rewrote it, tombi rewrote it
back and the hook reported a modified file.

Exclude uv.lock from tombi-format so uv stays authoritative for its
generated lockfile, and bump the workspace members to 0.7.0 so the lock
is current and uv stops re-resolving it (uv lock --check now passes).
2026-05-29 15:04:48 +02:00
Andrey Avtomonov
0a517b2c13
skill: document adding context sources; docs: one-shot full-demo path (#234)
- skills/ktx/SKILL.md: add an "Add context sources" section with the generic
  `ktx setup --source ...` flags per connector (dbt, Metabase, Notion, ...),
  warehouse mapping, the --metabase-database-id discovery note, and the
  `ktx ingest` follow-up. The skill previously only documented database setup
  with --skip-sources, so agents couldn't wire up dbt/Metabase/Notion (KLO-723).
- docs-site quickstart: the kaelio.com/start callout now points at the
  "copy agent setup" one-shot prompt that installs the full four-source demo.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 15:02:57 +02:00
Andrey Avtomonov
d53cdac366
chore: upgrade dependencies and tooling (#232)
* chore: upgrade dependencies and tooling

* chore: upgrade dependencies and tooling
2026-05-29 11:56:55 +02:00
559 changed files with 33796 additions and 10723 deletions

View file

@ -217,7 +217,7 @@ jobs:
flags: typescript flags: typescript
name: typescript name: typescript
disable_search: true disable_search: true
fail_ci_if_error: true fail_ci_if_error: false
- name: Warn when Codecov token is missing for TypeScript - name: Warn when Codecov token is missing for TypeScript
if: env.CODECOV_TOKEN_CONFIGURED != 'true' if: env.CODECOV_TOKEN_CONFIGURED != 'true'
@ -236,7 +236,7 @@ jobs:
flags: python flags: python
name: python name: python
disable_search: true disable_search: true
fail_ci_if_error: true fail_ci_if_error: false
- name: Warn when Codecov token is missing for Python - name: Warn when Codecov token is missing for Python
if: env.CODECOV_TOKEN_CONFIGURED != 'true' if: env.CODECOV_TOKEN_CONFIGURED != 'true'

72
.github/workflows/star-history.yml vendored Normal file
View file

@ -0,0 +1,72 @@
name: Refresh star history chart
on:
schedule:
# Twice daily at 06:00 and 18:00 UTC.
- cron: "0 6,18 * * *"
workflow_dispatch:
permissions:
contents: write
env:
DO_NOT_TRACK: "1"
KTX_TELEMETRY_DISABLED: "1"
NEXT_TELEMETRY_DISABLED: "1"
concurrency:
group: star-history-refresh
cancel-in-progress: true
jobs:
refresh:
name: Regenerate assets/star-history.svg
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# RELEASE_PAT can push to the protected main branch; the default
# GITHUB_TOKEN is rejected by the branch-protection hook (GH006).
token: ${{ secrets.RELEASE_PAT }}
- name: Fetch fresh star-history SVG
run: |
set -euo pipefail
# cachebust forces star-history to regenerate instead of serving its
# own server-side cache; --location follows the slug-normalizing 301.
url="https://api.star-history.com/svg?repos=Kaelio/ktx&type=Date&cachebust=${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
curl --fail --location --silent --show-error \
--retry 3 --retry-delay 5 --max-time 60 \
-o assets/star-history.svg.new "$url"
# Guard against error pages / truncated responses before overwriting.
if ! grep -q "</svg>" assets/star-history.svg.new; then
echo "Downloaded file is not a valid SVG; aborting." >&2
exit 1
fi
if [ "$(wc -c < assets/star-history.svg.new)" -lt 1000 ]; then
echo "Downloaded SVG is suspiciously small; aborting." >&2
exit 1
fi
# The star-history API returns the SVG without a trailing newline,
# which end-of-file-fixer rewrites whenever pre-commit runs
# --all-files on a PR. Because the refresh commit below uses [skip ci],
# the hook never runs against it here, so an un-normalized file
# silently breaks the pre-commit check on every open PR. Normalize to
# exactly one trailing newline before committing.
printf '%s\n' "$(cat assets/star-history.svg.new)" > assets/star-history.svg
rm -f assets/star-history.svg.new
- name: Commit if changed
run: |
set -euo pipefail
if git diff --quiet -- assets/star-history.svg; then
echo "Star-history chart unchanged; nothing to commit."
exit 0
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add assets/star-history.svg
# [skip ci] keeps this housekeeping commit from triggering KTX CI.
git commit -m "chore: refresh star history chart [skip ci]"
git push

View file

@ -22,7 +22,7 @@ jobs:
github.event.issue.author_association != 'COLLABORATOR' github.event.issue.author_association != 'COLLABORATOR'
steps: steps:
- name: Apply needs-triage label - name: Apply needs-triage label
uses: actions/github-script@v7 uses: actions/github-script@v9
with: with:
script: | script: |
await github.rest.issues.addLabels({ await github.rest.issues.addLabels({

View file

@ -14,6 +14,18 @@ repos:
- id: check-case-conflict - id: check-case-conflict
- id: mixed-line-ending - id: mixed-line-ending
- repo: https://github.com/tombi-toml/tombi-pre-commit
rev: v1.1.0
hooks:
- id: tombi-format
args: ["--offline"]
# uv.lock is generated and owned by uv, which writes its own canonical
# TOML layout. tombi reformats that layout differently, so once uv
# regenerates the lock (e.g. after a dependency or version change)
# tombi rewrites it and the hook fails on the modified file. Keep uv
# authoritative for its lockfile; tombi still formats hand-edited TOML.
exclude: ^uv\.lock$
- repo: https://github.com/asottile/pyupgrade - repo: https://github.com/asottile/pyupgrade
rev: v3.21.2 rev: v3.21.2
hooks: hooks:

153
AGENTS.md
View file

@ -64,6 +64,25 @@ When rules conflict, follow this order:
4. Code quality: types, readable boundaries, focused modules 4. Code quality: types, readable boundaries, focused modules
5. Performance where it matters 5. Performance where it matters
## Opinionated Product Defaults
- **MUST**: Prefer one canonical behavior over configurable alternatives. A new
flag, config field, environment variable, mode, strategy option, adapter hook,
or fallback path is a product feature and must be justified by an explicit
user request or a real correctness requirement.
- **MUST NOT**: Add speculative flexibility for imagined users, migrations,
review preferences, local workflows, or "just in case" scenarios. If the
requested behavior can work with one solid default, implement that default.
- **MUST NOT**: Add boolean switches that create two runtime paths unless both
paths are essential and the user explicitly asked for the choice. Boolean
policy knobs are especially suspect because they double the state space and
test surface.
- **MUST**: When a design seems to need a new option, first try to remove the
need by choosing the stronger default, tightening the invariant, or failing
clearly. Ask the user before adding the option if it still seems necessary.
- **MUST**: Delete obsolete branches, tests, docs, and config when removing a
behavior. Do not preserve dormant compatibility paths.
## Repository Shape ## Repository Shape
**ktx** is a pnpm + uv workspace. **ktx** is a pnpm + uv workspace.
@ -159,6 +178,111 @@ and naming asymmetries are bugs in waiting — see
[`docs/code-design.md`](docs/code-design.md). Treat the `MUST` / `MUST NOT` [`docs/code-design.md`](docs/code-design.md). Treat the `MUST` / `MUST NOT`
rules there with the same weight as the ones in this file. rules there with the same weight as the ones in this file.
## Design Reasoning Defaults
When proposing a design, an approach, or any non-trivial change, apply these
defaults and run the self-check before presenting it. They encode the
corrections users most often have to make; reaching these conclusions
autonomously — without being asked the leading question — is the bar.
- **MUST**: Optimize for the best outcome, not for an unstated constraint. Do not
silently adopt "smallest change", "least effort", "cheapest", or "least user
intervention" as the goal unless the user said so. Default to the most correct,
durable solution, and present cost / effort / scope as information for the user
to weigh — not as a ceiling you impose on their behalf.
- **MUST**: Separate one-time cost from recurring cost before discarding an
option. A fixed cost paid once (a setup-time computation, an extra LLM call
during setup, a contract change) to make every later run cheaper or more
correct is usually worth it. Do not reject it with recurring-cost reasoning;
quantify both sides. (Example smell: "don't add an LLM call to a cost-cutting
feature" — wrong when the call is one-time and the savings recur.)
- **MUST**: Treat a user's example as a representative of a class, not as the
spec. Design for the general population the example stands for, then stress-test
against deliberately different instances — another warehouse, dialect, stack
layout, or input shape — before committing. If a design only works because of an
incidental property of the example (e.g. "the noise happened to be in a separate
schema *on this demo*"), it is curve-fitting; generalize it or state the
assumption explicitly.
- **MUST**: Prefer deriving from the system's own state over enumerating cases.
Favor an allowlist computed from declared/observed state (config, scanned
catalog, query log, the user's own inputs) over a denylist of known-bad
specifics (particular tables, schemas, tools, or vendors). A hardcoded or
hand-maintained list of external specifics is a smell: it rots and fails on the
next stack. The only acceptable static patterns are genuinely universal
invariants (e.g. DB-engine system catalogs) and ktx's own self-emitted
signatures.
- **MUST**: Give each capability one implementation and route every caller
through it. When some behavior — running a query, resolving a credential or
config reference, authenticating, selecting a dialect, loading config —
already has a working implementation that some call sites use, make new or
divergent call sites depend on that path instead of standing up a second one.
Parallel implementations of one capability drift apart silently: a fix, a
newly supported input, or an added case lands on one path and not the other,
so one entry point (a CLI command, an MCP tool, an ingest stage) succeeds
while another fails on the same input. When two paths already do the same
job, collapse onto the shared one and delete the duplicate instead of
keeping both. When fixing a defect that lives on one path, fix the shared
implementation; do not patch the symptom on a forked branch, which preserves
the divergence you set out to remove.
- **SHOULD**: Before inventing an abstraction or hand-rolling structural logic,
search for what already exists and reuse it — the codebase's canonical
representation (a structured ref/key type) instead of a parallel string scheme,
and a mandated/available tool (e.g. `sqlglot` for SQL structure; see
[SQL and Structured Parsing](#sql-and-structured-parsing)) instead of
hand-parsing. Normalize ambiguous input to the canonical form at the boundary;
do not carry the ambiguity downstream. This is the single-source-of-truth / DRY
item from the Priority Hierarchy applied at design time.
Before presenting a design, answer these explicitly:
1. Am I optimizing for a goal the user actually stated, or one I assumed?
2. Does this generalize beyond the example in front of me? Name a real case where
it would break.
3. Am I enumerating known-bad cases when I could derive scope from the system's
own declared/observed state?
4. Is there an existing canonical representation or mandated tool I should reuse
instead of building or parsing my own?
5. Am I discarding the better option on a weak or misapplied constraint
(one-time vs recurring cost, "more surface area", "more work now")?
6. Does another entry point already perform this operation through a shared
implementation? If so, am I routing through that path instead of forking a
parallel one — and if I'm fixing a bug, am I fixing the shared layer rather
than one branch?
7. Am I adding a user-visible option or alternate runtime path that the user did
not ask for? If yes, can one opinionated default solve the problem instead?
8. Does this option multiply behavior by caller path, config value, or local
state? If yes, remove it unless it is explicitly required.
A user question that nudges toward an alternative ("would X help?", "should I
always do Y?", "will you hardcode Z?") is a signal that a better option exists.
Investigate the implied direction and reason it through *before* defending the
original proposal — and prefer to have asked yourself the question first.
Example: If generated context changes should be saved, choose one save policy
and route ingest, setup, memory, indexing, and docs through it. Do not add an
`auto_commit`-style switch unless the user explicitly asks for staged-only runs
and accepts the extra runtime path.
## Code Comments
Code must be self-explanatory. A comment exists only to state a constraint the
code cannot show; everything else belongs in the PR description or nowhere.
- **MUST**: Keep each comment to 1-3 lines stating only what the code cannot
show: a cross-file invariant ("error-severity issues never reach here — the
doctor exits on them first"), a required ordering ("ktx.yaml is written
before git init, so a crash cannot leave a bare `.git`"), or a library quirk
("zod reports unknown record keys as `invalid_key`").
- **MUST**: State each invariant once, at the public entry point. Do not repeat
the same guarantee across a helper, its wrapper, and the call site.
- **MUST NOT**: Write prose comment blocks — design rationale, alternatives
considered, change narration ("is now written before…"), caller enumerations
("shared by X, Y, and Z"), or restatements of what the code already shows.
That is the author addressing the reviewer, and it rots once merged.
- **MAY**: Open a regression test with a 1-3 line comment stating the scenario
it guards when the test name cannot carry it. Omit design history and
references to removed designs.
## TypeScript Standards ## TypeScript Standards
- Use Node 22+ and pnpm workspace commands. - Use Node 22+ and pnpm workspace commands.
@ -278,7 +402,8 @@ use `PascalCase` without the suffix.
## Telemetry ## Telemetry
**ktx** ships anonymous PostHog telemetry. When adding commands or events: **ktx** ships PostHog usage telemetry. Catalog telemetry events use strict
schemas. When adding commands or events:
- **MUST NOT**: Add fields that carry user data — file paths, hostnames, - **MUST NOT**: Add fields that carry user data — file paths, hostnames,
environment values, SQL text, schema/table/column names, error messages, environment values, SQL text, schema/table/column names, error messages,
@ -295,6 +420,24 @@ use `PascalCase` without the suffix.
of collected data changes. Adding another event with no new field types of collected data changes. Adding another event with no new field types
needs no docs change. needs no docs change.
### Error reports
**ktx** also sends PostHog Error Tracking `$exception` events when telemetry is
enabled. This channel is separate from the strict catalog event schema and is
used only for exception diagnostics.
`$exception` events may include stack frames, error class names, raw error
messages, cause chains, `source`, `handled`, `fatal`, runtime version fields,
OS/runtime fields, and the hashed `projectId` when known. Stack frames may
include local file paths and the local username when those appear in paths.
`$exception` events must never intentionally include secrets, credentials,
database URLs, auth headers, raw argv, raw environment values, SQL text,
schema/table/column names as explicit properties, customer row data, user prompt
text, or raw MCP arguments. Reporters must redact call-site-provided secret
snapshots and common static credential patterns before the SDK serializes the
exception.
## Documentation and Specs ## Documentation and Specs
- Keep public documentation in `README.md`, package READMEs, example READMEs, - Keep public documentation in `README.md`, package READMEs, example READMEs,
@ -350,8 +493,9 @@ error messages — including the disambiguation rule for the overloaded word
`source` (semantic / primary / context / source of truth) — see `source` (semantic / primary / context / source of truth) — see
[`docs/terminology.md`](docs/terminology.md). Follow that file when choosing [`docs/terminology.md`](docs/terminology.md). Follow that file when choosing
between near-synonyms (e.g. `connector` vs `adapter`, `data agent` vs between near-synonyms (e.g. `connector` vs `adapter`, `data agent` vs
`database agent`, `fast ingest` vs `schema ingest`). Product-name rules in `database agent`, `context-source ingest` vs `source ingest`). Product-name
this section take precedence over anything in that file when they conflict. rules in this section take precedence over anything in that file when they
conflict.
### Updating `docs-site/` After Code Changes ### Updating `docs-site/` After Code Changes
@ -380,7 +524,8 @@ rather than silently skipping it.
- **MUST**: Disable monospace ligatures on every surface that uses the - **MUST**: Disable monospace ligatures on every surface that uses the
`var(--font-mono)` family (Geist Mono). Geist Mono fuses `--` into an `var(--font-mono)` family (Geist Mono). Geist Mono fuses `--` into an
em-dash glyph that visually eats the adjacent space, so prompts like em-dash glyph that visually eats the adjacent space, so prompts like
`npx skills add Kaelio/ktx --skill ktx` render as `Kaelio/ktx--skill ktx`. `npx skills add Kaelio/ktx --skill ktx` render as
`Kaelio/ktx--skill ktx`.
- **MUST**: When adding a new container that renders user-visible monospace - **MUST**: When adding a new container that renders user-visible monospace
text outside `<code>` / `<pre>` (e.g. a styled `<div className="font-mono">` text outside `<code>` / `<pre>` (e.g. a styled `<div className="font-mono">`
for a copyable prompt), verify the global ligature-off rule in for a copyable prompt), verify the global ligature-off rule in

View file

@ -1,6 +1,6 @@
# Contributing to KTX # Contributing to ktx
Thanks for your interest in KTX. This page covers **how to contribute** and Thanks for your interest in **ktx**. This page covers **how to contribute** and
the **contributor rewards program**. For development setup, repository the **contributor rewards program**. For development setup, repository
layout, and verification commands, see the layout, and verification commands, see the
[Contributing guide in the docs](https://docs.kaelio.com/ktx/docs/community/contributing). [Contributing guide in the docs](https://docs.kaelio.com/ktx/docs/community/contributing).
@ -23,7 +23,7 @@ layout, and verification commands, see the
## Contributor rewards program ## Contributor rewards program
We send merch to contributors whose pull requests get merged. The goal is We send merch to contributors whose pull requests get merged. The goal is
to thank the people building KTX with us, not to drive volume. to thank the people building **ktx** with us, not to drive volume.
### How it works ### How it works
@ -76,7 +76,7 @@ See the [Community & Support](https://docs.kaelio.com/ktx/docs/community/support
page for the full guide. The short version: page for the full guide. The short version:
- **Questions, "how do I...", setup help, sharing patterns**: join the - **Questions, "how do I...", setup help, sharing patterns**: join the
[KTX Slack](https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ). [**ktx** Slack](https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ).
- **Bugs**: use the [Bug report](.github/ISSUE_TEMPLATE/bug_report.yml) - **Bugs**: use the [Bug report](.github/ISSUE_TEMPLATE/bug_report.yml)
template. template.
- **Feature requests**: use the - **Feature requests**: use the
@ -87,7 +87,7 @@ page for the full guide. The short version:
## Code of conduct ## Code of conduct
KTX follows the **ktx** follows the
[Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/). [Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/).
Be respectful, assume good intent, and keep discussion focused on the Be respectful, assume good intent, and keep discussion focused on the
project. Report concerns to the maintainers in Slack or by email at project. Report concerns to the maintainers in Slack or by email at

View file

@ -13,16 +13,20 @@
<a href="https://docs.kaelio.com/ktx/docs/"><img src="https://img.shields.io/badge/docs-ktx-22c55e?style=flat-square" alt="Documentation" /></a> <a href="https://docs.kaelio.com/ktx/docs/"><img src="https://img.shields.io/badge/docs-ktx-22c55e?style=flat-square" alt="Documentation" /></a>
<a href="https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ"><img src="https://img.shields.io/badge/slack-join%20community-4A154B?style=flat-square&logo=slack&logoColor=white" alt="Join the ktx Slack community" /></a> <a href="https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ"><img src="https://img.shields.io/badge/slack-join%20community-4A154B?style=flat-square&logo=slack&logoColor=white" alt="Join the ktx Slack community" /></a>
<a href="https://github.com/Kaelio/ktx/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-Apache%202.0-blue?style=flat-square" alt="License" /></a> <a href="https://github.com/Kaelio/ktx/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-Apache%202.0-blue?style=flat-square" alt="License" /></a>
<a href="https://www.ycombinator.com/companies?batch=P25"><img src="https://img.shields.io/badge/Y%20Combinator-P25-orange?style=flat-square" alt="Y Combinator P25" /></a> <a href="https://www.ycombinator.com/companies/kaelio"><img src="https://img.shields.io/badge/Y%20Combinator-P25-orange?style=flat-square" alt="Y Combinator P25" /></a>
</p> </p>
<p align="center"> <p align="center">
<a href="https://docs.kaelio.com/ktx/docs/getting-started/quickstart"><b>Quickstart</b></a> · <a href="https://docs.kaelio.com/ktx/docs/getting-started/quickstart"><b>Quickstart</b></a> ·
<a href="https://docs.kaelio.com/ktx/docs/cli-reference/ktx"><b>CLI Reference</b></a> · <a href="https://docs.kaelio.com/ktx/docs/cli-reference/ktx"><b>CLI Reference</b></a> ·
<a href="https://docs.kaelio.com/ktx/docs/ai-resources/agent-quickstart"><b>Agent Setup</b></a> · <a href="https://docs.kaelio.com/ktx/docs/community/ai-resources"><b>Agent Setup</b></a> ·
<a href="https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ"><b>Slack</b></a> <a href="https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ"><b>Slack</b></a>
</p> </p>
<p align="center">
<sub>Built and maintained by <a href="https://www.kaelio.com"><b>Kaelio</b></a></sub>
</p>
--- ---
**ktx** is a self-improving context layer that teaches agents how to query your **ktx** is a self-improving context layer that teaches agents how to query your
@ -30,13 +34,25 @@ warehouse accurately - from approved metric definitions, joinable columns, and
business knowledge it builds and maintains for you. business knowledge it builds and maintains for you.
> [!NOTE] > [!NOTE]
> Run **ktx** with your own LLM API keys or a **Claude Pro/Max** subscription. > Run **ktx** with your own LLM API keys or a local agent sign-in — a
> No extra usage billing from **ktx**. > **Claude Pro/Max** subscription through Claude Code, or your local Codex
> authentication. No extra usage billing from **ktx**.
<p align="center"> <p align="center">
<img src="docs-site/public/images/ingestion-flow-transparent.svg" alt="ktx ingestion flow from source systems through validation to wiki and semantic-layer outputs" width="900" /> <a href="https://youtu.be/5V4TuzYVlrA">
<img src="assets/launch-video-thumb.png" alt="Watch the ktx launch video (1:56)" width="820" />
</a>
</p> </p>
<p align="center">
<img src="docs-site/public/images/ingestion-flow.png" alt="Ingestion: ktx ingests databases, BI tools, modeling code, and docs through its context engine (source connectors, context builder, reconciliation, validation) into wiki Markdown and semantic-layer YAML" width="900" />
</p>
<p align="center">
<img src="docs-site/public/images/mcp-runtime-flow.png" alt="Serving: an agent queries ktx through MCP, which searches the wiki and semantic layer, returns approved metrics, and compiles them into read-only SQL run against the warehouse" width="900" />
</p>
## Why ktx ## Why ktx
General-purpose agents struggle on data tasks. They re-explore your warehouse General-purpose agents struggle on data tasks. They re-explore your warehouse
@ -127,6 +143,14 @@ Agent integration ready: yes (codex:project)
> If `ktx status` prints `ktx mcp start --project-dir ...`, run it before > If `ktx status` prints `ktx mcp start --project-dir ...`, run it before
> opening your agent client. > opening your agent client.
## Upgrading
Re-run the global install with the `@latest` tag:
```bash
npm install -g @kaelio/ktx@latest
```
## First commands ## First commands
| Command | Purpose | | Command | Purpose |
@ -164,8 +188,9 @@ then the current directory. Pass `--project-dir <path>` when scripting.
No. **ktx** runs locally. The only data leaving your machine is what you No. **ktx** runs locally. The only data leaving your machine is what you
send to the LLM provider you configured. send to the LLM provider you configured.
- **Which LLM backends are supported?** - **Which LLM backends are supported?**
Anthropic API, Google Vertex AI, AI Gateway, and the local Claude Code Anthropic API, Google Vertex AI, AI Gateway, the local Claude Code session
session through the Claude Agent SDK. See through the Claude Agent SDK, and your local Codex authentication through the
Codex SDK. See
[LLM configuration](https://docs.kaelio.com/ktx/docs/guides/llm-configuration). [LLM configuration](https://docs.kaelio.com/ktx/docs/guides/llm-configuration).
- **How is ktx different from a dbt or MetricFlow semantic layer?** - **How is ktx different from a dbt or MetricFlow semantic layer?**
**ktx** *ingests* those layers and combines them with raw-table **ktx** *ingests* those layers and combines them with raw-table
@ -184,7 +209,7 @@ then the current directory. Pass `--project-dir <path>` when scripting.
- [The Context Layer](https://docs.kaelio.com/ktx/docs/concepts/the-context-layer) - [The Context Layer](https://docs.kaelio.com/ktx/docs/concepts/the-context-layer)
- [Building Context](https://docs.kaelio.com/ktx/docs/guides/building-context) - [Building Context](https://docs.kaelio.com/ktx/docs/guides/building-context)
- [CLI Reference](https://docs.kaelio.com/ktx/docs/cli-reference/ktx) - [CLI Reference](https://docs.kaelio.com/ktx/docs/cli-reference/ktx)
- [Agent Quickstart](https://docs.kaelio.com/ktx/docs/ai-resources/agent-quickstart) - [AI Resources](https://docs.kaelio.com/ktx/docs/community/ai-resources)
- [Community & Support](https://docs.kaelio.com/ktx/docs/community/support) - [Community & Support](https://docs.kaelio.com/ktx/docs/community/support)
## Community ## Community
@ -234,11 +259,17 @@ uv run pytest -q
## Telemetry ## Telemetry
**ktx** collects anonymous usage telemetry from interactive CLI runs to **ktx** collects privacy-conscious usage telemetry to understand installs and
improve setup, command reliability, and data-agent workflows. No file paths, improve setup, command reliability, and data-agent workflows. Catalog telemetry
hostnames, SQL, schema names, error messages, or argv are recorded. See events do not record file paths, hostnames, SQL, schema names, table names,
[Telemetry](https://docs.kaelio.com/ktx/docs/community/telemetry) for the column names, error messages, raw environment values, or argv. Error reports use
event catalog and opt-out options. PostHog Error Tracking and can include stack frames and raw error messages,
which may contain local file paths or the local username in those paths.
**ktx** redacts secrets, credentials, database URLs, auth headers, argv, raw
environment values, SQL text, row data, and user-typed prompt or MCP argument
text from the explicit `$exception` payload. See
[Telemetry](https://docs.kaelio.com/ktx/docs/community/telemetry) for the event
catalog and opt-out options.
## License ## License
@ -248,6 +279,6 @@ event catalog and opt-out options.
<p align="center"> <p align="center">
<a href="https://star-history.com/#Kaelio/ktx&Date"> <a href="https://star-history.com/#Kaelio/ktx&Date">
<img src="https://api.star-history.com/svg?repos=Kaelio/ktx&type=Date" alt="ktx Star History Chart" width="700" /> <img src="assets/star-history.svg" alt="ktx Star History Chart" width="700" />
</a> </a>
</p> </p>

View file

@ -2,20 +2,20 @@
## Reporting a vulnerability ## Reporting a vulnerability
If you believe you've found a security vulnerability in KTX, please report it If you believe you've found a security vulnerability in **ktx**, please report it
**privately** through GitHub Security Advisories: **privately** through GitHub Security Advisories:
[Report a vulnerability](https://github.com/Kaelio/ktx/security/advisories/new) [Report a vulnerability](https://github.com/Kaelio/ktx/security/advisories/new)
If you cannot use GitHub Security Advisories, email `support@kaelio.com` If you cannot use GitHub Security Advisories, email `support@kaelio.com`
instead. Please do **not** open a public issue, post in the KTX Slack, or instead. Please do **not** open a public issue, post in the **ktx** Slack, or
share details elsewhere until we have published a fix. share details elsewhere until we have published a fix.
When reporting, please include: When reporting, please include:
- A description of the issue and its impact - A description of the issue and its impact
- Steps to reproduce - Steps to reproduce
- The KTX version affected - The **ktx** version affected
## What to expect ## What to expect

View file

@ -19,14 +19,9 @@
<path d="M 80 84 Q 86 77 92 84" fill="none" stroke="#F5F1EA" stroke-width="3.5" stroke-linecap="round" /> <path d="M 80 84 Q 86 77 92 84" fill="none" stroke="#F5F1EA" stroke-width="3.5" stroke-linecap="round" />
<path d="M 108 84 Q 114 77 120 84" fill="none" stroke="#F5F1EA" stroke-width="3.5" stroke-linecap="round" /> <path d="M 108 84 Q 114 77 120 84" fill="none" stroke="#F5F1EA" stroke-width="3.5" stroke-linecap="round" />
<!-- wordmark: 'ktx', half the logo height, vertically centered --> <!-- wordmark: "ktx" outlined from Outfit SemiBold (the docs-site display font)
<text so it renders identically everywhere, independent of installed fonts -->
x="225" <g transform="translate(242 145)" fill="#1B3139">
y="145" <path d="M51.17 0 25.06 -34.79 51.03 -67.62H72.17L41.65 -30.7L42.35 -39.55L73.57 0ZM8.05 0V-101.22H26.46V0ZM88.41 0V-95.69H106.82V0ZM72.66 -51.52V-67.62H122.57V-51.52ZM171.75 0 153.93 -27.41 150.22 -30.17 123.83 -67.62H145.64L161.91 -42.77L165.48 -40.18L193.38 0ZM122.54 0 150.05 -38.61 160.62 -26.22 143.19 0ZM166.11 -30.38 155.44 -42.67 171.54 -67.62H192.08Z" />
font-family="'JetBrains Mono', 'Fira Code', ui-monospace, 'SF Mono', Menlo, monospace" </g>
font-size="140"
font-weight="600"
fill="#1B3139"
letter-spacing="-0.04em"
>ktx</text>
</svg> </svg>

Before

Width:  |  Height:  |  Size: 1.1 KiB

After

Width:  |  Height:  |  Size: 1.4 KiB

Before After
Before After

Binary file not shown.

After

Width:  |  Height:  |  Size: 135 KiB

1
assets/star-history.svg Normal file

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 63 KiB

View file

@ -0,0 +1,12 @@
import type { Metadata } from "next";
import { DiagramStudio } from "@/components/diagram-studio/studio";
export const metadata: Metadata = {
title: "Diagram studio",
robots: { index: false, follow: false },
};
export default function DiagramStudioPage() {
return <DiagramStudio />;
}

View file

@ -869,6 +869,97 @@ body::after {
50% { opacity: 0.65; transform: scale(0.9); } 50% { opacity: 0.65; transform: scale(0.9); }
} }
/*
GitHub star widget (sidebar footer pill)
Rendered as the `icon` of a fumadocs icon-link, so it sits in the footer
pill beside the Slack mark and the theme toggle. GitHub mark + star glyph
+ live count; the star rotates to coral on hover. The !important sizes win
over fumadocs' `[&_svg]:size-4.5` rule on the wrapping link.
*/
.ktx-stars {
display: inline-flex;
align-items: center;
gap: 6px;
font-family: var(--font-display), var(--font-sans), sans-serif;
font-size: 13px;
line-height: 1;
}
/* Push the stars to the opposite (right) end of the footer pill, leaving the
Slack mark on the left like justify-content: space-between. The auto margin
absorbs the pill's free space; we cancel the theme toggle's own ms-auto so
that single gap lands before the stars, not between stars and the toggle. */
#nd-sidebar a[aria-label="Star ktx on GitHub"] {
margin-inline-start: auto;
}
#nd-sidebar [data-theme-toggle] {
margin-inline-start: 0;
}
.ktx-stars-gh {
width: 16px !important;
height: 16px !important;
flex-shrink: 0;
}
.ktx-stars-count-wrap {
display: inline-flex;
align-items: center;
gap: 4px;
}
.ktx-stars-star {
width: 12px !important;
height: 12px !important;
flex-shrink: 0;
fill: currentColor;
opacity: 0.7;
transition:
transform 0.3s var(--ktx-ease),
fill 0.3s var(--ktx-ease),
opacity 0.3s var(--ktx-ease);
}
/* The wrapping fumadocs link owns the hover; rotate + colour the star from it. */
#nd-sidebar a:hover .ktx-stars-star {
transform: rotate(-14deg) scale(1.12);
fill: var(--ktx-coral);
opacity: 1;
}
.ktx-stars-count {
font-weight: 600;
font-variant-numeric: tabular-nums;
letter-spacing: -0.01em;
}
/* Skeleton shown only on the rare cold (uncached) fetch */
.ktx-stars-skeleton-bar {
display: inline-block;
width: 26px;
height: 10px;
border-radius: 4px;
background: linear-gradient(
90deg,
var(--color-fd-muted) 25%,
color-mix(in oklch, var(--color-fd-muted-foreground) 28%, var(--color-fd-muted)) 50%,
var(--color-fd-muted) 75%
);
background-size: 200% 100%;
animation: ktx-stars-shimmer 1.4s ease-in-out infinite;
}
@keyframes ktx-stars-shimmer {
from { background-position: 200% 0; }
to { background-position: -200% 0; }
}
@media (prefers-reduced-motion: reduce) {
#nd-sidebar a:hover .ktx-stars-star { transform: none; }
.ktx-stars-skeleton-bar { animation: none; }
}
/* Dot grid */ /* Dot grid */
.dot-grid { .dot-grid {
background-image: radial-gradient( background-image: radial-gradient(

View file

@ -1,22 +1,22 @@
import type { BaseLayoutProps } from "fumadocs-ui/layouts/shared"; import type { BaseLayoutProps } from "fumadocs-ui/layouts/shared";
import { GitHubIcon } from "@/components/github-icon";
import { Logo } from "@/components/logo"; import { Logo } from "@/components/logo";
import { SlackIcon } from "@/components/slack-icon"; import { SlackIcon } from "@/components/slack-icon";
import { GitHubStars, GITHUB_REPO_URL } from "@/components/github-stars";
import { ThemeToggle } from "@/components/theme-toggle";
export const baseOptions: BaseLayoutProps = { export const baseOptions: BaseLayoutProps = {
nav: { nav: {
title: <Logo />, title: Logo,
transparentMode: "top", transparentMode: "top",
}, },
links: [ // Custom two-icon switcher (light / dark) where each icon selects its own
{ // theme. The default "light-dark" switcher is a single blind toggle — both
type: "icon", // icons just flip the theme, so clicking the sun while already in light mode
label: "GitHub", // jumps to dark, which reads as broken.
icon: <GitHubIcon />, slots: {
text: "GitHub", themeSwitch: ThemeToggle,
url: "https://github.com/kaelio/ktx",
external: true,
}, },
links: [
{ {
type: "icon", type: "icon",
label: "Join the ktx Slack community", label: "Join the ktx Slack community",
@ -25,5 +25,13 @@ export const baseOptions: BaseLayoutProps = {
url: "https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ", url: "https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ",
external: true, external: true,
}, },
{
type: "icon",
label: "Star ktx on GitHub",
icon: <GitHubStars />,
text: "GitHub",
url: GITHUB_REPO_URL,
external: true,
},
], ],
}; };

View file

@ -0,0 +1,328 @@
import { type Edge, MarkerType, type Node } from "@xyflow/react";
import { C } from "./nodes";
const EDGE_COLOR = "#b3bcc4";
const MARKER_COLOR = "#9aa6ad";
const labelStyle = {
fontFamily: "var(--font-inter), system-ui, sans-serif",
fontSize: 15,
fontWeight: 600,
fill: C.inkMuted,
};
const labelBgStyle = { fill: "#ffffff", stroke: C.chipBorder, strokeWidth: 1 };
const labelBg = {
labelBgPadding: [8, 4] as [number, number],
labelBgBorderRadius: 6,
labelStyle,
labelBgStyle,
};
const marker = { type: MarkerType.ArrowClosed, color: MARKER_COLOR, width: 16, height: 16 };
const edgeStyle = { stroke: EDGE_COLOR, strokeWidth: 2 };
/* ============================== INGESTION =============================== */
const SRC_W = 300;
const SRC_H = 138;
const SRC_GAP = 24;
const srcY = (i: number) => i * (SRC_H + SRC_GAP);
export const ingestionNodes: Node[] = [
{
id: "title",
type: "title",
position: { x: 0, y: -96 },
data: {
width: 560,
eyebrow: "1 · Ingestion",
title: "ktx builds your context layer",
},
},
{
id: "db",
type: "card",
position: { x: 0, y: srcY(0) },
data: {
width: SRC_W,
height: SRC_H,
accent: C.teal,
rows: [
{ kind: "title", text: "Databases" },
{ kind: "desc", text: "Schemas, keys, query history" },
{ kind: "muted", text: "Postgres · Snowflake · BigQuery · …" },
],
handles: [{ side: "right", type: "source", id: "out" }],
},
},
{
id: "bi",
type: "card",
position: { x: 0, y: srcY(1) },
data: {
width: SRC_W,
height: SRC_H,
accent: C.orange,
rows: [
{ kind: "title", text: "BI tools" },
{ kind: "desc", text: "Dashboards, explores, usage" },
{ kind: "muted", text: "Metabase · Looker · …" },
],
handles: [{ side: "right", type: "source", id: "out" }],
},
},
{
id: "model",
type: "card",
position: { x: 0, y: srcY(2) },
data: {
width: SRC_W,
height: SRC_H,
accent: C.amber,
rows: [
{ kind: "title", text: "Modeling code" },
{ kind: "desc", text: "Metrics, models, joins, entities" },
{ kind: "muted", text: "dbt · LookML · MetricFlow · …" },
],
handles: [{ side: "right", type: "source", id: "out" }],
},
},
{
id: "docs",
type: "card",
position: { x: 0, y: srcY(3) },
data: {
width: SRC_W,
height: SRC_H,
accent: C.emerald,
rows: [
{ kind: "title", text: "Docs & notes" },
{ kind: "desc", text: "Policies, definitions, notes" },
{ kind: "muted", text: "Notion · any text · …" },
],
handles: [{ side: "right", type: "source", id: "out" }],
},
},
{
id: "engine",
type: "engine",
position: { x: 420, y: 52 },
data: {
width: 380,
height: 520,
steps: [
{ n: 1, title: "Source connectors", desc: "Read each source in its shape" },
{ n: 2, title: "Context builder", desc: "Evidence into proposed updates" },
{ n: 3, title: "Reconciliation", desc: "Merge with existing context" },
{ n: 4, title: "Validation", desc: "Check references & semantics" },
],
handles: [
{ side: "left", type: "target", id: "in" },
{ side: "right", type: "source", id: "out" },
],
},
},
{
id: "wiki",
type: "card",
position: { x: 900, y: 66 },
data: {
width: 320,
height: 220,
accent: C.emerald,
rows: [
{ kind: "mono", text: "wiki/*.md", color: C.emerald },
{ kind: "title", text: "Wiki" },
{ kind: "chips", items: ["free-form", "auto-maintained"] },
{ kind: "desc", text: "Definitions, caveats, policies," },
{ kind: "desc", text: "and notes agents can search." },
],
handles: [{ side: "left", type: "target", id: "in" }],
},
},
{
id: "sl",
type: "card",
position: { x: 900, y: 338 },
data: {
width: 320,
height: 220,
accent: C.teal,
rows: [
{ kind: "mono", text: "semantic-layer/*.yaml", color: C.teal },
{ kind: "title", text: "Semantic layer" },
{ kind: "chips", items: ["executable", "auto-maintained"] },
{ kind: "desc", text: "Metrics, joins, dimensions, and" },
{ kind: "desc", text: "filters ktx compiles into SQL." },
],
handles: [{ side: "left", type: "target", id: "in" }],
},
},
];
const ingestEdge = (source: string, target: string): Edge => ({
id: `${source}-${target}`,
source,
target,
sourceHandle: "out",
targetHandle: "in",
type: "default",
style: edgeStyle,
markerEnd: marker,
});
export const ingestionEdges: Edge[] = [
ingestEdge("db", "engine"),
ingestEdge("bi", "engine"),
ingestEdge("model", "engine"),
ingestEdge("docs", "engine"),
ingestEdge("engine", "wiki"),
ingestEdge("engine", "sl"),
];
/* =============================== RUNTIME ================================ */
export const runtimeNodes: Node[] = [
{
id: "title",
type: "title",
position: { x: 0, y: -84 },
data: {
width: 560,
eyebrow: "2 · Serving",
title: "agents query it through MCP",
},
},
{
id: "agent",
type: "card",
position: { x: 0, y: 115 },
data: {
width: 280,
height: 190,
accent: C.neutral,
align: "center",
rows: [
{ kind: "title", text: "Your agent" },
{ kind: "muted", text: "Claude Code · Cursor" },
{ kind: "muted", text: "Codex · OpenCode" },
],
handles: [
{ side: "right", type: "source", id: "ask", top: "42%" },
{ side: "right", type: "target", id: "answer", top: "62%" },
],
},
},
{
id: "hub",
type: "hub",
position: { x: 420, y: 85 },
data: {
width: 360,
height: 250,
rows: [
"Search wiki + semantic layer",
"Return approved metrics",
"Compile metrics → SQL",
],
handles: [
{ side: "left", type: "target", id: "ask", top: "42%" },
{ side: "left", type: "source", id: "answer", top: "62%" },
{ side: "right", type: "source", id: "to-context", top: "30%" },
{ side: "right", type: "source", id: "to-warehouse", top: "72%" },
],
},
},
{
id: "context",
type: "card",
position: { x: 920, y: 15 },
data: {
width: 300,
height: 150,
accent: C.teal,
rows: [
{ kind: "title", text: "Context layer" },
{ kind: "mono", text: "wiki/*.md", color: C.emerald },
{ kind: "mono", text: "semantic-layer/*.yaml", color: C.teal },
],
handles: [{ side: "left", type: "target", id: "in" }],
},
},
{
id: "warehouse",
type: "card",
position: { x: 920, y: 255 },
data: {
width: 300,
height: 150,
accent: C.slate,
rows: [
{ kind: "title", text: "Warehouse" },
{
kind: "badge",
text: "read-only",
bg: "#ecf6f8",
border: "#bfe3ea",
color: C.teal,
},
{ kind: "desc", text: "Runs the compiled SQL" },
],
handles: [{ side: "left", type: "target", id: "in" }],
},
},
];
export const runtimeEdges: Edge[] = [
{
id: "ask",
source: "agent",
sourceHandle: "ask",
target: "hub",
targetHandle: "ask",
type: "default",
label: "ask",
...labelBg,
style: edgeStyle,
markerEnd: marker,
},
{
id: "answer",
source: "hub",
sourceHandle: "answer",
target: "agent",
targetHandle: "answer",
type: "default",
label: "answer",
...labelBg,
style: edgeStyle,
markerEnd: marker,
},
{
id: "search",
source: "hub",
sourceHandle: "to-context",
target: "context",
targetHandle: "in",
type: "smoothstep",
label: "search + read",
...labelBg,
style: edgeStyle,
markerStart: marker,
markerEnd: marker,
},
{
id: "readonly",
source: "hub",
sourceHandle: "to-warehouse",
target: "warehouse",
targetHandle: "in",
type: "smoothstep",
label: "read-only",
...labelBg,
style: edgeStyle,
markerStart: marker,
markerEnd: marker,
},
];

View file

@ -0,0 +1,57 @@
/**
* Inlined ktx mascot, ported from assets/ktx-mascot.svg.
*
* - `light` renders the dark-bodied mascot for light surfaces.
* - `dark` renders the cream-bodied mascot for dark surfaces (e.g. the ktx
* hub panel), mirroring brand/ktx-mascot-dark.svg.
*/
export function KtxMascot({
variant = "light",
size = 56,
}: {
variant?: "light" | "dark";
size?: number;
}) {
const body = variant === "dark" ? "#F5F1EA" : "#1B3139";
const eye = variant === "dark" ? "#1B3139" : "#F5F1EA";
return (
<svg
viewBox="0 0 200 200"
width={size}
height={size}
role="img"
aria-label="ktx mascot"
>
<g fill="none" stroke={body} strokeWidth="16" strokeLinecap="round">
<path d="M 62 110 Q 32 130 44 152" />
<path d="M 88 116 Q 80 152 70 174" />
<path d="M 112 116 Q 120 152 130 174" />
</g>
<path
d="M 134 108 C 162 116, 172 96, 162 78 C 154 64, 168 56, 178 60"
fill="none"
stroke="#FF8A4C"
strokeWidth="16"
strokeLinecap="round"
/>
<path
d="M 48 102 C 48 56, 78 30, 100 30 C 122 30, 152 56, 152 102 C 152 116, 132 120, 100 120 C 68 120, 48 116, 48 102 Z"
fill={body}
/>
<path
d="M 80 84 Q 86 77 92 84"
fill="none"
stroke={eye}
strokeWidth="3.5"
strokeLinecap="round"
/>
<path
d="M 108 84 Q 114 77 120 84"
fill="none"
stroke={eye}
strokeWidth="3.5"
strokeLinecap="round"
/>
</svg>
);
}

View file

@ -0,0 +1,493 @@
"use client";
import { Handle, Position, type Node, type NodeProps } from "@xyflow/react";
import { KtxMascot } from "./mascot";
/** Fixed palette mirrored from the approved SVG diagrams so the exported PNG
* is theme-independent (one image that reads on light and dark GitHub). */
export const C = {
ink: "#1b1b18",
inkSoft: "#57534e",
inkMuted: "#8c857f",
cardBorder: "#e2dfd9",
engineBg: "#15323a",
engineBorder: "#23474f",
cyan: "#55dced",
stepNum: "#06262c",
stepTitle: "#f3f1ec",
stepDesc: "#9fb6bc",
hubRow: "#eef4f5",
chipBg: "#faf9f6",
chipBorder: "#e7e5e4",
teal: "#0e7490",
emerald: "#059669",
orange: "#f97316",
amber: "#d97706",
slate: "#334155",
neutral: "#94a3b8",
} as const;
const DISPLAY = "var(--font-display), system-ui, sans-serif";
const BODY = "var(--font-inter), system-ui, sans-serif";
const MONO = "var(--font-mono), ui-monospace, monospace";
const CARD_SHADOW = "0 3px 12px rgba(27, 49, 57, 0.10)";
const ENGINE_SHADOW = "0 6px 22px rgba(2, 12, 15, 0.30)";
/** ktx logo mascot size, shared by the engine and hub headers. */
const LOGO_SIZE = 56;
type HandleSpec = {
side: "left" | "right";
type: "source" | "target";
id: string;
top?: string;
};
function Handles({ specs }: { specs?: HandleSpec[] }) {
if (!specs) return null;
return (
<>
{specs.map((h) => (
<Handle
key={`${h.type}-${h.id}`}
id={h.id}
type={h.type}
position={h.side === "left" ? Position.Left : Position.Right}
isConnectable={false}
style={{
opacity: 0,
border: 0,
background: "transparent",
...(h.top ? { top: h.top } : {}),
}}
/>
))}
</>
);
}
/* ------------------------------- Card node ------------------------------- */
type CardRow =
| { kind: "title"; text: string }
| { kind: "mono"; text: string; color: string }
| { kind: "desc"; text: string }
| { kind: "muted"; text: string }
| { kind: "chips"; items: string[] }
| { kind: "badge"; text: string; bg: string; border: string; color: string };
type CardData = {
width: number;
height: number;
accent: string;
align?: "center";
rows: CardRow[];
handles?: HandleSpec[];
};
function gapFor(kind: CardRow["kind"], prev?: CardRow["kind"]): number {
if (!prev) return 0;
if (kind === "desc" && prev === "desc") return 3;
if (kind === "mono" && prev === "mono") return 2;
if (kind === "title") return 6;
return 10;
}
function CardRowView({ row }: { row: CardRow }) {
switch (row.kind) {
case "title":
return (
<span
style={{
fontFamily: DISPLAY,
fontWeight: 700,
fontSize: 26,
lineHeight: 1.15,
color: C.ink,
}}
>
{row.text}
</span>
);
case "mono":
return (
<span
style={{
fontFamily: MONO,
fontWeight: 700,
fontSize: 18,
lineHeight: 1.4,
color: row.color,
}}
>
{row.text}
</span>
);
case "desc":
return (
<span
style={{
fontFamily: BODY,
fontWeight: 500,
fontSize: 17,
lineHeight: 1.45,
color: C.inkSoft,
}}
>
{row.text}
</span>
);
case "muted":
return (
<span
style={{
fontFamily: BODY,
fontWeight: 500,
fontSize: 14,
lineHeight: 1.4,
color: C.inkMuted,
}}
>
{row.text}
</span>
);
case "chips":
return (
<div style={{ display: "flex", gap: 8, flexWrap: "wrap" }}>
{row.items.map((c) => (
<span
key={c}
style={{
fontFamily: BODY,
fontWeight: 600,
fontSize: 14,
color: C.inkSoft,
background: C.chipBg,
border: `1px solid ${C.chipBorder}`,
borderRadius: 6,
padding: "4px 10px",
}}
>
{c}
</span>
))}
</div>
);
case "badge":
return (
<span
style={{
display: "inline-flex",
alignItems: "center",
borderRadius: 14,
padding: "3px 12px",
fontFamily: BODY,
fontWeight: 700,
fontSize: 14,
background: row.bg,
border: `1px solid ${row.border}`,
color: row.color,
}}
>
{row.text}
</span>
);
}
}
function CardNode({ data }: NodeProps<Node<CardData>>) {
const center = data.align === "center";
return (
<div
style={{
width: data.width,
height: data.height,
position: "relative",
background: "#ffffff",
border: `1px solid ${C.cardBorder}`,
borderRadius: 10,
boxShadow: CARD_SHADOW,
padding: "18px 20px",
display: "flex",
flexDirection: "column",
alignItems: center ? "center" : "flex-start",
justifyContent: center ? "center" : "flex-start",
textAlign: center ? "center" : "left",
overflow: "hidden",
}}
>
<span
style={{
position: "absolute",
top: 0,
left: 2,
right: 2,
height: 4,
borderRadius: 2,
background: data.accent,
}}
/>
<Handles specs={data.handles} />
{data.rows.map((row, i) => (
<div
key={i}
style={{ marginTop: gapFor(row.kind, data.rows[i - 1]?.kind) }}
>
<CardRowView row={row} />
</div>
))}
</div>
);
}
/* ------------------------------ Engine node ------------------------------ */
type EngineStep = { n: number; title: string; desc: string };
type EngineData = {
width: number;
height: number;
steps: EngineStep[];
handles?: HandleSpec[];
};
function EngineNode({ data }: NodeProps<Node<EngineData>>) {
return (
<div
style={{
width: data.width,
height: data.height,
position: "relative",
background: C.engineBg,
border: `1px solid ${C.engineBorder}`,
borderRadius: 14,
boxShadow: ENGINE_SHADOW,
padding: "24px 24px",
display: "flex",
flexDirection: "column",
overflow: "hidden",
}}
>
<span
style={{
position: "absolute",
top: 0,
left: 2,
right: 2,
height: 4,
borderRadius: 2,
background: C.cyan,
}}
/>
<Handles specs={data.handles} />
<div style={{ display: "flex", alignItems: "center", gap: 14 }}>
<KtxMascot variant="dark" size={LOGO_SIZE} />
<span
style={{
fontFamily: DISPLAY,
fontWeight: 700,
fontSize: 30,
color: C.stepTitle,
}}
>
ktx
</span>
</div>
<div
style={{
flex: 1,
display: "flex",
flexDirection: "column",
justifyContent: "space-around",
marginTop: 6,
}}
>
{data.steps.map((s) => (
<div
key={s.n}
style={{ display: "flex", alignItems: "center", gap: 18 }}
>
<span
style={{
flex: "none",
width: 44,
height: 44,
borderRadius: "50%",
background: C.cyan,
display: "flex",
alignItems: "center",
justifyContent: "center",
fontFamily: DISPLAY,
fontWeight: 800,
fontSize: 22,
color: C.stepNum,
}}
>
{s.n}
</span>
<div style={{ display: "flex", flexDirection: "column", gap: 3 }}>
<span
style={{
fontFamily: DISPLAY,
fontWeight: 700,
fontSize: 24,
lineHeight: 1.1,
color: C.stepTitle,
}}
>
{s.title}
</span>
<span
style={{
fontFamily: BODY,
fontWeight: 500,
fontSize: 16,
lineHeight: 1.3,
color: C.stepDesc,
}}
>
{s.desc}
</span>
</div>
</div>
))}
</div>
</div>
);
}
/* -------------------------------- Hub node ------------------------------- */
type HubData = {
width: number;
height: number;
rows: string[];
handles?: HandleSpec[];
};
function HubNode({ data }: NodeProps<Node<HubData>>) {
return (
<div
style={{
width: data.width,
height: data.height,
position: "relative",
background: C.engineBg,
border: `1px solid ${C.engineBorder}`,
borderRadius: 14,
boxShadow: ENGINE_SHADOW,
padding: "24px 24px",
display: "flex",
flexDirection: "column",
overflow: "hidden",
}}
>
<span
style={{
position: "absolute",
top: 0,
left: 2,
right: 2,
height: 4,
borderRadius: 2,
background: C.cyan,
}}
/>
<Handles specs={data.handles} />
<div style={{ display: "flex", alignItems: "center", gap: 14 }}>
<KtxMascot variant="dark" size={LOGO_SIZE} />
<span
style={{
fontFamily: DISPLAY,
fontWeight: 700,
fontSize: 30,
color: C.stepTitle,
}}
>
ktx
</span>
</div>
<div
style={{
marginTop: 22,
display: "flex",
flexDirection: "column",
gap: 18,
}}
>
{data.rows.map((r) => (
<div key={r} style={{ display: "flex", alignItems: "center", gap: 14 }}>
<span
style={{
flex: "none",
width: 10,
height: 10,
borderRadius: "50%",
background: C.cyan,
}}
/>
<span
style={{
fontFamily: BODY,
fontWeight: 600,
fontSize: 19,
color: C.hubRow,
}}
>
{r}
</span>
</div>
))}
</div>
</div>
);
}
/* ------------------------------- Title node ------------------------------ */
type TitleData = { width: number; eyebrow: string; title: string };
function TitleNode({ data }: NodeProps<Node<TitleData>>) {
return (
<div
style={{
width: data.width,
display: "flex",
flexDirection: "column",
gap: 6,
}}
>
<span
style={{
fontFamily: BODY,
fontSize: 19,
fontWeight: 800,
letterSpacing: 2,
textTransform: "uppercase",
color: C.teal,
}}
>
{data.eyebrow}
</span>
<span
style={{
fontFamily: DISPLAY,
fontSize: 24,
fontWeight: 600,
color: C.inkMuted,
}}
>
{data.title}
</span>
</div>
);
}
export const nodeTypes = {
card: CardNode,
engine: EngineNode,
hub: HubNode,
title: TitleNode,
};

View file

@ -0,0 +1,242 @@
"use client";
import "@xyflow/react/dist/style.css";
import { useCallback, useRef, useState } from "react";
import {
Background,
BackgroundVariant,
type Edge,
getNodesBounds,
type Node,
ReactFlow,
ReactFlowProvider,
useEdgesState,
useNodesState,
useReactFlow,
} from "@xyflow/react";
import { toPng } from "html-to-image";
import {
ingestionEdges,
ingestionNodes,
runtimeEdges,
runtimeNodes,
} from "./flows";
import { nodeTypes } from "./nodes";
const EXPORT_PADDING = 48;
const EXPORT_PIXEL_RATIO = 2;
function DiagramCanvasInner({
initialNodes,
initialEdges,
fileName,
height,
dark,
}: {
initialNodes: Node[];
initialEdges: Edge[];
fileName: string;
height: number;
dark: boolean;
}) {
const wrapperRef = useRef<HTMLDivElement>(null);
const [nodes, , onNodesChange] = useNodesState(initialNodes);
const [edges, , onEdgesChange] = useEdgesState(initialEdges);
const { getNodes } = useReactFlow();
const [busy, setBusy] = useState(false);
const download = useCallback(async () => {
const viewport = wrapperRef.current?.querySelector<HTMLElement>(
".react-flow__viewport",
);
if (!viewport) return;
setBusy(true);
try {
await document.fonts.ready;
const bounds = getNodesBounds(getNodes());
const outW = Math.ceil(bounds.width + EXPORT_PADDING * 2);
const outH = Math.ceil(bounds.height + EXPORT_PADDING * 2);
const tx = EXPORT_PADDING - bounds.x;
const ty = EXPORT_PADDING - bounds.y;
const dataUrl = await toPng(viewport, {
width: outW,
height: outH,
pixelRatio: EXPORT_PIXEL_RATIO,
// transparent background so one PNG works on light and dark GitHub
style: {
width: `${outW}px`,
height: `${outH}px`,
transform: `translate(${tx}px, ${ty}px) scale(1)`,
},
});
const link = document.createElement("a");
link.download = fileName;
link.href = dataUrl;
link.click();
} finally {
setBusy(false);
}
}, [fileName, getNodes]);
return (
<div>
<div style={{ display: "flex", gap: 8, marginBottom: 10 }}>
<button
type="button"
onClick={download}
disabled={busy}
style={btnStyle(busy)}
>
{busy ? "Exporting…" : "Download PNG"}
</button>
</div>
<div
ref={wrapperRef}
style={{
height,
borderRadius: 12,
border: "1px solid rgba(127,127,127,0.2)",
background: dark ? "#0d1117" : "#ffffff",
}}
>
<ReactFlow
nodes={nodes}
edges={edges}
nodeTypes={nodeTypes}
onNodesChange={onNodesChange}
onEdgesChange={onEdgesChange}
fitView
fitViewOptions={{ padding: 0.08 }}
nodesDraggable={false}
nodesConnectable={false}
nodesFocusable={false}
edgesFocusable={false}
elementsSelectable={false}
panOnDrag={false}
panOnScroll={false}
zoomOnScroll={false}
zoomOnPinch={false}
zoomOnDoubleClick={false}
preventScrolling={false}
proOptions={{ hideAttribution: true }}
>
<Background
variant={BackgroundVariant.Dots}
gap={18}
size={1}
color={dark ? "#1f2a30" : "#e6e2db"}
/>
</ReactFlow>
</div>
</div>
);
}
function btnStyle(disabled: boolean): React.CSSProperties {
return {
fontFamily: "var(--font-inter), system-ui, sans-serif",
fontSize: 13,
fontWeight: 600,
padding: "7px 14px",
borderRadius: 8,
border: "1px solid #0e7490",
background: disabled ? "#9bbdc6" : "#0e7490",
color: "#ffffff",
cursor: disabled ? "default" : "pointer",
};
}
function DiagramCanvas(props: {
initialNodes: Node[];
initialEdges: Edge[];
fileName: string;
height: number;
dark: boolean;
}) {
return (
<ReactFlowProvider>
<DiagramCanvasInner {...props} />
</ReactFlowProvider>
);
}
export function DiagramStudio() {
const [dark, setDark] = useState(false);
return (
<main
style={{
maxWidth: 1320,
margin: "0 auto",
padding: "32px 24px 80px",
fontFamily: "var(--font-inter), system-ui, sans-serif",
}}
>
<header style={{ marginBottom: 24 }}>
<h1
style={{
fontFamily: "var(--font-display), system-ui, sans-serif",
fontSize: 30,
fontWeight: 700,
color: "#1b1b18",
margin: 0,
}}
>
ktx diagram studio
</h1>
<p style={{ color: "#6b6560", marginTop: 6, fontSize: 15 }}>
Static diagrams. Export is a transparent 2× PNG framed to the node
bounds the dark-background toggle is only for previewing.
</p>
<label
style={{
display: "inline-flex",
alignItems: "center",
gap: 8,
marginTop: 12,
fontSize: 14,
color: "#57534e",
}}
>
<input
type="checkbox"
checked={dark}
onChange={(e) => setDark(e.target.checked)}
/>
Preview on dark background
</label>
</header>
<section style={{ marginBottom: 40 }}>
<h2 style={sectionTitle}>1 · Ingestion building the context layer</h2>
<DiagramCanvas
initialNodes={ingestionNodes}
initialEdges={ingestionEdges}
fileName="ingestion-flow.png"
height={560}
dark={dark}
/>
</section>
<section>
<h2 style={sectionTitle}>2 · Serving answering agents at runtime</h2>
<DiagramCanvas
initialNodes={runtimeNodes}
initialEdges={runtimeEdges}
fileName="mcp-runtime-flow.png"
height={480}
dark={dark}
/>
</section>
</main>
);
}
const sectionTitle: React.CSSProperties = {
fontFamily: "var(--font-display), system-ui, sans-serif",
fontSize: 18,
fontWeight: 600,
color: "#1b1b18",
marginBottom: 12,
};

View file

@ -0,0 +1,82 @@
import { Suspense } from "react";
import { GitHubIcon } from "@/components/github-icon";
const REPO = "kaelio/ktx";
export const GITHUB_REPO_URL = `https://github.com/${REPO}`;
const API_URL = `https://api.github.com/repos/${REPO}`;
async function fetchStarCount(): Promise<number | null> {
try {
const res = await fetch(API_URL, {
headers: { Accept: "application/vnd.github+json" },
// Revalidate hourly. GitHub's unauthenticated REST limit is 60 req/h per
// IP, so a single cached server-side fetch keeps the count fresh while
// never exposing visitors to rate limits or layout shift.
next: { revalidate: 3600 },
});
if (!res.ok) return null;
const data = (await res.json()) as { stargazers_count?: unknown };
return typeof data.stargazers_count === "number"
? data.stargazers_count
: null;
} catch {
return null;
}
}
/** Compact, GitHub-style count: 847 → "847", 1234 → "1.2k", 12345 → "12.3k". */
function formatStars(count: number): string {
if (count < 1000) return count.toLocaleString("en-US");
const thousands = count / 1000;
const rounded =
thousands >= 100 ? Math.round(thousands) : Math.round(thousands * 10) / 10;
return `${rounded}k`;
}
function StarGlyph() {
return (
<svg className="ktx-stars-star" viewBox="0 0 24 24" aria-hidden="true">
<path d="M12 2.6l2.9 5.88 6.49.95-4.7 4.57 1.11 6.46L12 17.4l-5.8 3.06 1.11-6.46-4.7-4.57 6.49-.95z" />
</svg>
);
}
async function StarsInner() {
const count = await fetchStarCount();
return (
<span className="ktx-stars">
<GitHubIcon className="ktx-stars-gh" />
{count !== null ? (
<span className="ktx-stars-count-wrap">
<StarGlyph />
<span className="ktx-stars-count">{formatStars(count)}</span>
</span>
) : (
<span className="ktx-stars-count">Star</span>
)}
</span>
);
}
function StarsSkeleton() {
return (
<span className="ktx-stars" aria-hidden="true">
<GitHubIcon className="ktx-stars-gh" />
<span className="ktx-stars-skeleton-bar" />
</span>
);
}
/**
* Footer star widget GitHub mark + live count. Rendered as the `icon` of a
* fumadocs `type: "icon"` link, so it lands in the sidebar footer pill beside
* the Slack icon and the theme toggle. fumadocs supplies the surrounding <a>
* (href + aria-label), so this renders inner content only no anchor.
*/
export function GitHubStars() {
return (
<Suspense fallback={<StarsSkeleton />}>
<StarsInner />
</Suspense>
);
}

View file

@ -1,7 +1,17 @@
export function Logo() { "use client";
import Link from "next/link";
const brandFont = {
fontFamily: "var(--font-display), var(--font-sans), sans-serif",
} as const;
export function Logo({ href = "/", className }: { href?: string; className?: string }) {
return ( return (
<div className={className}>
<div className="flex items-center gap-3.5 group"> <div className="flex items-center gap-3.5 group">
<div className="relative flex items-center justify-center transition-transform duration-300 ease-out group-hover:rotate-[-4deg]"> <Link href={href} aria-label="ktx documentation home" className="flex items-center no-underline">
<span className="relative flex items-center justify-center transition-transform duration-300 ease-out group-hover:rotate-[-4deg]">
<img <img
src="/ktx/brand/ktx-mascot.svg" src="/ktx/brand/ktx-mascot.svg"
alt="" alt=""
@ -14,27 +24,33 @@ export function Logo() {
aria-hidden="true" aria-hidden="true"
className="h-20 w-20 object-contain hidden dark:block" className="h-20 w-20 object-contain hidden dark:block"
/> />
</div> </span>
</Link>
<div className="flex flex-col items-start leading-none"> <div className="flex flex-col items-start leading-none">
<span <Link
className="text-[42px] font-semibold text-fd-foreground tracking-tight" href={href}
style={{ fontFamily: "var(--font-display), var(--font-sans), sans-serif" }} className="text-[42px] font-semibold text-fd-foreground tracking-tight no-underline"
style={brandFont}
> >
ktx ktx
</span> </Link>
<span <a
className="mt-1 whitespace-nowrap text-[13px] font-medium text-fd-muted-foreground/80 tracking-tight" href="https://www.kaelio.com"
style={{ fontFamily: "var(--font-display), var(--font-sans), sans-serif" }} target="_blank"
rel="noreferrer"
className="mt-1 whitespace-nowrap text-[13px] font-medium text-fd-muted-foreground/80 tracking-tight no-underline transition-colors hover:text-fd-foreground"
style={brandFont}
> >
by Kaelio by Kaelio
</span> </a>
</div> </div>
<span <span
className="text-[19px] font-medium text-fd-muted-foreground/80 tracking-tight border-l border-fd-border pl-3 ml-1" className="text-[19px] font-medium text-fd-muted-foreground/80 tracking-tight border-l border-fd-border pl-3 ml-1"
style={{ fontFamily: "var(--font-display), var(--font-sans), sans-serif" }} style={brandFont}
> >
Docs Docs
</span> </span>
</div> </div>
</div>
); );
} }

View file

@ -0,0 +1,576 @@
"use client";
import {
type Edge,
type EdgeProps,
getSmoothStepPath,
Handle,
MarkerType,
type Node,
type NodeProps,
Position,
} from "@xyflow/react";
import { FlowCanvas } from "./flow-canvas";
type AgentNodeData = {
title: string;
items: string[];
};
type HubNodeData = {
title: string;
badge: string;
rows: string[];
};
type TargetNodeData = {
accent: string;
title: string;
body: string;
rows: { text: string; color?: string; mono?: boolean }[];
badge?: string;
};
type AgentNode = Node<AgentNodeData, "agent">;
type HubNode = Node<HubNodeData, "hub">;
type TargetNode = Node<TargetNodeData, "target">;
type FlowNode = AgentNode | HubNode | TargetNode;
const AGENT_W = 252;
const AGENT_H = 96;
const HUB_W = 306;
const HUB_H = 190;
const TARGET_W = 268;
const TARGET_H = 148;
const CENTER_X = 470;
const ROW_AGENT_Y = 0;
const ROW_HUB_Y = 196;
const ROW_TARGET_Y = 488;
const AGENT_X = CENTER_X - AGENT_W / 2;
const HUB_X = CENTER_X - HUB_W / 2;
const TARGET_GAP_X = 38;
const TARGETS_TOTAL = TARGET_W * 2 + TARGET_GAP_X;
const TARGETS_START_X = CENTER_X - TARGETS_TOTAL / 2;
const CONTEXT_X = TARGETS_START_X;
const WAREHOUSE_X = TARGETS_START_X + TARGET_W + TARGET_GAP_X;
const EDGE_STROKE = "#94a3b8";
const CYCLE_STROKE = "#0e7490";
const EMERALD = "#059669";
const TEAL = "#0e7490";
const nodes: FlowNode[] = [
{
id: "agent",
type: "agent",
position: { x: AGENT_X, y: ROW_AGENT_Y },
data: {
title: "Your agent",
items: ["Claude Code", "Cursor", "Codex"],
},
draggable: false,
selectable: false,
},
{
id: "hub",
type: "hub",
position: { x: HUB_X, y: ROW_HUB_Y },
data: {
title: "ktx",
badge: "MCP + CLI",
rows: [
"Search wiki + semantic layer",
"Return approved metrics",
"Compile metrics → SQL",
],
},
draggable: false,
selectable: false,
},
{
id: "context",
type: "target",
position: { x: CONTEXT_X, y: ROW_TARGET_Y },
data: {
accent: TEAL,
title: "Context layer",
body: "Approved definitions agents search before they answer.",
rows: [
{ text: "wiki/*.md", color: EMERALD, mono: true },
{ text: "semantic-layer/*.yaml", color: TEAL, mono: true },
],
},
draggable: false,
selectable: false,
},
{
id: "warehouse",
type: "target",
position: { x: WAREHOUSE_X, y: ROW_TARGET_Y },
data: {
accent: "#334155",
title: "Database",
badge: "read-only",
body: "Runs the compiled SQL. ktx never writes to it.",
rows: [],
},
draggable: false,
selectable: false,
},
];
const labelBg = {
labelBgPadding: [6, 3] as [number, number],
labelBgBorderRadius: 4,
labelStyle: {
fontSize: 13,
fontWeight: 600,
fill: "var(--color-fd-muted-foreground)",
},
labelBgStyle: {
fill: "var(--color-fd-background)",
stroke: "var(--color-fd-border)",
strokeWidth: 1,
},
};
const requestMarker = {
type: MarkerType.ArrowClosed,
color: EDGE_STROKE,
width: 16,
height: 16,
};
const flowEdges: Edge[] = [
{
id: "e-ask",
source: "agent",
sourceHandle: "ask",
target: "hub",
targetHandle: "ask",
type: "straight",
label: "ask",
...labelBg,
style: { stroke: EDGE_STROKE, strokeWidth: 1.5 },
markerEnd: requestMarker,
},
{
id: "e-answer",
source: "hub",
sourceHandle: "answer",
target: "agent",
targetHandle: "answer",
type: "straight",
label: "answer",
...labelBg,
style: { stroke: EDGE_STROKE, strokeWidth: 1.5 },
markerEnd: requestMarker,
},
{
id: "e-search",
source: "hub",
sourceHandle: "to-context",
target: "context",
targetHandle: "in",
type: "smoothstep",
label: "search + read",
...labelBg,
style: { stroke: CYCLE_STROKE, strokeWidth: 1.5 },
markerStart: { type: MarkerType.ArrowClosed, color: CYCLE_STROKE, width: 14, height: 14 },
markerEnd: { type: MarkerType.ArrowClosed, color: CYCLE_STROKE, width: 14, height: 14 },
},
{
id: "e-readonly",
source: "hub",
sourceHandle: "to-warehouse",
target: "warehouse",
targetHandle: "in",
type: "smoothstep",
label: "read-only",
...labelBg,
style: { stroke: CYCLE_STROKE, strokeWidth: 1.5 },
markerStart: { type: MarkerType.ArrowClosed, color: CYCLE_STROKE, width: 14, height: 14 },
markerEnd: { type: MarkerType.ArrowClosed, color: CYCLE_STROKE, width: 14, height: 14 },
},
];
function AgentNodeView({ data }: NodeProps<AgentNode>) {
return (
<div
style={{ width: AGENT_W, height: AGENT_H }}
className="flex flex-col justify-center rounded-md border border-fd-border bg-fd-card px-3.5 py-2.5 shadow-sm"
>
<Handle
id="ask"
type="source"
position={Position.Bottom}
className="!opacity-0"
style={{ left: "35%" }}
/>
<Handle
id="answer"
type="target"
position={Position.Bottom}
className="!opacity-0"
style={{ left: "65%" }}
/>
<div className="flex items-center gap-2.5">
<span className="flex h-8 w-8 flex-none items-center justify-center rounded-full bg-fd-primary/15 text-fd-primary">
<svg
xmlns="http://www.w3.org/2000/svg"
width="18"
height="18"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="1.75"
strokeLinecap="round"
strokeLinejoin="round"
aria-hidden="true"
>
<rect x="3" y="6" width="18" height="12" rx="3" />
<circle cx="9" cy="12" r="1.25" fill="currentColor" stroke="none" />
<circle cx="15" cy="12" r="1.25" fill="currentColor" stroke="none" />
<path d="M12 3v3" />
</svg>
</span>
<p className="text-[17px] font-semibold leading-6 text-fd-foreground">
{data.title}
</p>
</div>
<div className="mt-2 flex flex-wrap gap-1.5">
{data.items.map((item) => (
<span
key={item}
className="rounded border border-fd-border bg-fd-background px-1.5 py-0.5 text-[12px] leading-5 text-fd-muted-foreground"
>
{item}
</span>
))}
</div>
</div>
);
}
function HubNodeView({ data }: NodeProps<HubNode>) {
return (
<div
style={{ width: HUB_W, height: HUB_H }}
className="relative flex flex-col rounded-md border border-cyan-200/20 bg-[#0f1f23] px-4 py-3.5 text-white shadow-sm dark:bg-[#0b181b]"
>
<Handle
id="ask"
type="target"
position={Position.Top}
className="!opacity-0"
style={{ left: "37.5%" }}
/>
<Handle
id="answer"
type="source"
position={Position.Top}
className="!opacity-0"
style={{ left: "62.5%" }}
/>
<Handle
id="to-context"
type="source"
position={Position.Bottom}
className="!opacity-0"
style={{ left: "44%" }}
/>
<Handle
id="to-warehouse"
type="source"
position={Position.Bottom}
className="!opacity-0"
style={{ left: "56%" }}
/>
<div className="flex items-center gap-2.5">
<span className="flex h-7 w-7 flex-none items-center justify-center rounded-md bg-cyan-300/95 font-mono text-sm font-bold text-[#0b1c20]">
k
</span>
<span className="text-[19px] font-bold leading-6 text-white">
{data.title}
</span>
<span className="ml-1 rounded border border-cyan-200/30 bg-white/5 px-1.5 py-0.5 font-mono text-[11px] leading-5 text-cyan-100/85">
{data.badge}
</span>
</div>
<div className="mt-3 flex flex-1 flex-col justify-center gap-2">
{data.rows.map((row) => (
<div key={row} className="flex items-center gap-2.5">
<span className="h-1.5 w-1.5 flex-none rounded-full bg-cyan-300/95" />
<span className="text-[14px] font-medium leading-5 text-cyan-50/90">
{row}
</span>
</div>
))}
</div>
</div>
);
}
function TargetNodeView({ data }: NodeProps<TargetNode>) {
return (
<div
style={{
width: TARGET_W,
height: TARGET_H,
borderTop: `3px solid ${data.accent}`,
}}
className="overflow-hidden rounded-md border border-fd-border bg-fd-card px-3.5 py-3 shadow-sm"
>
<Handle id="in" type="target" position={Position.Top} className="!opacity-0" />
<div className="flex items-center gap-2">
<p className="text-[17px] font-semibold leading-6 text-fd-foreground">
{data.title}
</p>
{data.badge ? (
<span
className="rounded-full px-1.5 py-0.5 text-[11px] font-semibold leading-5"
style={{
color: data.accent,
background: "color-mix(in oklch, var(--color-fd-card) 86%, #64748b)",
}}
>
{data.badge}
</span>
) : null}
</div>
{data.rows.length > 0 ? (
<div className="mt-1 flex flex-col gap-0.5">
{data.rows.map((row) => (
<span
key={row.text}
className={
row.mono
? "font-mono text-[13px] font-semibold tracking-tight"
: "text-[12px] leading-4 text-fd-muted-foreground"
}
style={row.color ? { color: row.color } : undefined}
>
{row.text}
</span>
))}
</div>
) : null}
<p className="mt-1.5 line-clamp-2 text-[13px] leading-[18px] text-fd-muted-foreground">
{data.body}
</p>
</div>
);
}
/* ------------------------------- Particles ------------------------------- */
const PARTICLE_SPEED_PX_PER_SEC = 150;
const PARTICLE_MIN_DURATION_SEC = 5;
type Leg = {
sx: number;
sy: number;
sPos: Position;
tx: number;
ty: number;
tPos: Position;
};
const AGENT_ASK_X = AGENT_X + AGENT_W * 0.35;
const AGENT_ANSWER_X = AGENT_X + AGENT_W * 0.65;
const AGENT_BOTTOM_Y = ROW_AGENT_Y + AGENT_H;
const HUB_ASK_X = HUB_X + HUB_W * 0.375;
const HUB_ANSWER_X = HUB_X + HUB_W * 0.625;
const HUB_TO_CONTEXT_X = HUB_X + HUB_W * 0.44;
const HUB_TO_WAREHOUSE_X = HUB_X + HUB_W * 0.56;
const HUB_BOTTOM_Y = ROW_HUB_Y + HUB_H;
const CONTEXT_TOP_X = CONTEXT_X + TARGET_W / 2;
const WAREHOUSE_TOP_X = WAREHOUSE_X + TARGET_W / 2;
function buildCyclePath(spokeX: number, targetX: number): {
d: string;
length: number;
} {
const legs: Leg[] = [
// agent → hub (ask, down)
{ sx: AGENT_ASK_X, sy: AGENT_BOTTOM_Y, sPos: Position.Bottom, tx: HUB_ASK_X, ty: ROW_HUB_Y, tPos: Position.Top },
// through the hub to its spoke handle (down, drawn behind the hub)
{ sx: HUB_ASK_X, sy: ROW_HUB_Y, sPos: Position.Bottom, tx: spokeX, ty: HUB_BOTTOM_Y, tPos: Position.Top },
// hub → target (down)
{ sx: spokeX, sy: HUB_BOTTOM_Y, sPos: Position.Bottom, tx: targetX, ty: ROW_TARGET_Y, tPos: Position.Top },
// target → hub (up)
{ sx: targetX, sy: ROW_TARGET_Y, sPos: Position.Top, tx: spokeX, ty: HUB_BOTTOM_Y, tPos: Position.Bottom },
// through the hub to its answer handle (up, drawn behind the hub)
{ sx: spokeX, sy: HUB_BOTTOM_Y, sPos: Position.Top, tx: HUB_ANSWER_X, ty: ROW_HUB_Y, tPos: Position.Bottom },
// hub → agent (answer, up)
{ sx: HUB_ANSWER_X, sy: ROW_HUB_Y, sPos: Position.Top, tx: AGENT_ANSWER_X, ty: AGENT_BOTTOM_Y, tPos: Position.Bottom },
];
const segments = legs.map((leg) => {
const [segment] = getSmoothStepPath({
sourceX: leg.sx,
sourceY: leg.sy,
sourcePosition: leg.sPos,
targetX: leg.tx,
targetY: leg.ty,
targetPosition: leg.tPos,
});
return segment;
});
let d = segments[0];
for (let i = 1; i < segments.length; i += 1) {
d += ` ${segments[i].replace(/^M/, "L")}`;
}
const length = legs.reduce(
(sum, leg) => sum + Math.abs(leg.tx - leg.sx) + Math.abs(leg.ty - leg.sy),
0,
);
return { d, length };
}
type ParticleEdgeData = {
d: string;
duration: number;
beginOffset: number;
color: string;
};
type ParticleEdge = Edge<ParticleEdgeData, "particle">;
function ParticleEdgeView({ id, data }: EdgeProps<ParticleEdge>) {
if (!data) return null;
const pathId = `runtime-particle-path-${id}`;
return (
<>
<path id={pathId} d={data.d} fill="none" stroke="none" pointerEvents="none" />
<g className="runtime-particle" style={{ color: data.color }}>
<circle r={7.5} fill="currentColor" opacity={0.16} />
<circle r={3.75} fill="currentColor" opacity={0.32} />
<circle r={2.1} fill="currentColor" />
<animateMotion
dur={`${data.duration.toFixed(2)}s`}
begin={`-${data.beginOffset.toFixed(2)}s`}
repeatCount="indefinite"
>
<mpath href={`#${pathId}`} />
</animateMotion>
</g>
</>
);
}
function makeCycleEdge(
id: string,
source: string,
spokeX: number,
targetX: number,
beginFraction: number,
): ParticleEdge {
const { d, length } = buildCyclePath(spokeX, targetX);
const duration = Math.max(
PARTICLE_MIN_DURATION_SEC,
length / PARTICLE_SPEED_PX_PER_SEC,
);
return {
id,
source,
target: source,
type: "particle",
data: { d, duration, beginOffset: duration * beginFraction, color: CYCLE_STROKE },
};
}
const particleEdges: ParticleEdge[] = [
makeCycleEdge("p-context", "context", HUB_TO_CONTEXT_X, CONTEXT_TOP_X, 0),
makeCycleEdge("p-warehouse", "warehouse", HUB_TO_WAREHOUSE_X, WAREHOUSE_TOP_X, 0.5),
];
const nodeTypes = {
agent: AgentNodeView,
hub: HubNodeView,
target: TargetNodeView,
};
const edgeTypes = {
particle: ParticleEdgeView,
};
const edges = [...flowEdges, ...particleEdges];
export function ProductRuntime() {
return (
<section
className="not-prose my-12 w-full max-w-full min-w-0 space-y-5"
aria-labelledby="runtime-title"
>
<div className="max-w-3xl">
<h2
id="runtime-title"
className="text-xl font-semibold tracking-normal text-fd-foreground sm:text-2xl"
style={{ fontFamily: "var(--font-display)" }}
>
How serving works
</h2>
<p className="mt-3 text-sm leading-6 text-fd-muted-foreground">
At runtime, agents reach ktx through MCP. ktx searches the context
layer, returns approved metrics, and compiles them into read-only SQL
the warehouse runs.
</p>
</div>
<article
className="max-w-full min-w-0 overflow-hidden rounded-lg border border-fd-border bg-fd-card shadow-sm"
aria-label="ktx serving flow from an agent request to a governed answer"
>
<div className="border-b border-fd-border bg-fd-muted/35 px-5 py-4">
<p className="text-xs font-semibold uppercase tracking-wide text-fd-primary">
Serving flow
</p>
<h3
className="mt-1 text-base font-semibold tracking-normal text-fd-foreground sm:text-lg"
style={{ fontFamily: "var(--font-display)" }}
>
From an agent request to a governed answer
</h3>
<p className="mt-2 max-w-3xl text-xs leading-5 text-fd-muted-foreground">
The agent asks in plain language. ktx is the only thing that touches
the context layer and the warehouse, and every database connection
is read-only.
</p>
</div>
<FlowCanvas
nodes={nodes}
edges={edges}
nodeTypes={nodeTypes}
edgeTypes={edgeTypes}
canvasStyle={{
height: "min(620px, 98vw)",
minHeight: 430,
}}
className="runtime-canvas"
fitViewOptions={{ padding: 0.06 }}
ariaLabel="ktx serving flow diagram"
/>
</article>
<style>{`
.runtime-canvas .runtime-particle {
pointer-events: none;
filter: drop-shadow(0 0 6px currentColor);
}
@media (prefers-reduced-motion: reduce) {
.runtime-canvas .runtime-particle {
display: none;
}
}
`}</style>
</section>
);
}

View file

@ -0,0 +1,97 @@
"use client";
import { useEffect, useState, type ComponentProps, type SVGProps } from "react";
import { useTheme } from "fumadocs-ui/provider/base";
/**
* Two-icon theme switcher (light / dark), each icon selecting its own theme
* unlike fumadocs' default "light-dark" switcher, which is a single blind
* toggle that flips on any click. Dropped into the sidebar footer pill via
* `slots.themeSwitch`, so fumadocs passes the container className (left
* divider, `ms-auto`, rounded inner buttons); we merge it onto our own base.
*
* Icons are inlined (the project doesn't depend on `lucide-react` directly);
* `useTheme` is re-exported by fumadocs so we avoid a bare `next-themes` import.
*/
function SunIcon(props: SVGProps<SVGSVGElement>) {
return (
<svg
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth={2}
strokeLinecap="round"
strokeLinejoin="round"
aria-hidden="true"
{...props}
>
<circle cx="12" cy="12" r="4" />
<path d="M12 2v2" />
<path d="M12 20v2" />
<path d="m4.93 4.93 1.41 1.41" />
<path d="m17.66 17.66 1.41 1.41" />
<path d="M2 12h2" />
<path d="M20 12h2" />
<path d="m6.34 17.66-1.41 1.41" />
<path d="m19.07 4.93-1.41 1.41" />
</svg>
);
}
function MoonIcon(props: SVGProps<SVGSVGElement>) {
return (
<svg
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth={2}
strokeLinecap="round"
strokeLinejoin="round"
aria-hidden="true"
{...props}
>
<path d="M12 3a6 6 0 0 0 9 9 9 9 0 1 1-9-9Z" />
</svg>
);
}
const OPTIONS = [
["light", SunIcon],
["dark", MoonIcon],
] as const;
function cx(...classes: (string | false | undefined)[]): string {
return classes.filter(Boolean).join(" ");
}
export function ThemeToggle({ className, ...props }: ComponentProps<"div">) {
const { setTheme, resolvedTheme } = useTheme();
const [mounted, setMounted] = useState(false);
useEffect(() => setMounted(true), []);
const active = mounted ? resolvedTheme : null;
return (
<div
className={cx("inline-flex items-center overflow-hidden border", className)}
data-theme-toggle=""
{...props}
>
{OPTIONS.map(([key, Icon]) => (
<button
key={key}
type="button"
aria-label={key}
onClick={() => setTheme(key)}
className={cx(
"size-6.5 p-1.5 transition-colors",
active === key
? "bg-fd-accent text-fd-accent-foreground"
: "text-fd-muted-foreground hover:text-fd-accent-foreground",
)}
>
<Icon className="size-full" />
</button>
))}
</div>
);
}

View file

@ -1,40 +0,0 @@
---
title: Agent Instructions
description: Suggested instructions for coding assistants that need to read and cite ktx docs.
---
Use these instructions when a coding assistant needs to answer questions from the **ktx** documentation.
```text
When answering ktx docs questions:
1. Start with https://docs.kaelio.com/ktx/llms.txt.
2. Fetch the smallest relevant Markdown page from the index.
3. Prefer /docs/<path>.md over rendered HTML.
4. Use https://docs.kaelio.com/ktx/llms-full.txt only when the task needs broad docs context.
5. Quote commands exactly from docs pages.
6. If docs and local repository behavior disagree, say what differs and prefer local verified output for code changes.
```
## What this is for
This page is for documentation consumption only:
- answering questions about **ktx**
- finding the right docs page
- citing setup or CLI guidance
- helping an assistant avoid stale or invented commands
It does not describe local tool configuration.
## Minimal project prompt
```text
You are helping with ktx. Read https://docs.kaelio.com/ktx/llms.txt first, then fetch only the Markdown pages needed for the task. Do not scrape the rendered docs site when a .md route exists.
```
## Repository prompt
```text
Before editing ktx docs, read /llms.txt and the affected .md docs pages. Keep AI Resources focused on docs consumption. After editing, verify /llms.txt, /llms-full.txt, and any changed .md routes.
```

View file

@ -1,54 +0,0 @@
---
title: Agent Quickstart
description: A task-first route for coding agents that need to understand ktx docs.
---
This page is for coding assistants reading or citing the **ktx** docs. It is intentionally limited to documentation lookup, docs navigation, and safe command discovery.
For Markdown endpoints, use [Markdown Access](/docs/ai-resources/markdown-access).
For reusable task prompts, use [Prompt Recipes](/docs/ai-resources/prompt-recipes).
To install **ktx** into an agent client, use [Agent Clients](/docs/integrations/agent-clients).
## First read
Agents should start with the smallest source that answers the task:
1. [`/llms.txt`](/llms.txt) - discover the docs and preferred entry points.
2. The relevant per-page Markdown URL, for example `/docs/getting-started/quickstart.md`.
3. [`/llms-full.txt`](/llms-full.txt) - use only when the task needs broad context across many pages.
## Task router
| User asks the agent to explain... | Read first | Then read |
|------------------------------------|------------|-----------|
| What **ktx** does | [Introduction](/docs/getting-started/introduction) | [The Context Layer](/docs/concepts/the-context-layer) |
| How to start from a checkout | [Quickstart](/docs/getting-started/quickstart) | [ktx setup](/docs/cli-reference/ktx-setup) |
| How to check project readiness | [ktx status](/docs/cli-reference/ktx-status) | [Quickstart](/docs/getting-started/quickstart) |
| How context gets built | [Building Context](/docs/guides/building-context) | [ktx ingest](/docs/cli-reference/ktx-ingest) |
| How semantic YAML works | [Writing Context](/docs/guides/writing-context) | [ktx sl](/docs/cli-reference/ktx-sl) |
| How machine-readable CLI output is shaped | [ktx sl](/docs/cli-reference/ktx-sl) | [ktx wiki](/docs/cli-reference/ktx-wiki) |
## Operating workflow
Use this workflow when the user asks an assistant to answer a **ktx** docs question:
1. Read [`/llms.txt`](/llms.txt).
2. Pick the smallest relevant `.md` page.
3. Use [`/llms-full.txt`](/llms-full.txt) only if the answer needs multiple sections of the docs.
4. Quote commands exactly from the docs page.
5. If a command affects a local project, ask the user before assuming credentials or live services are available.
## Docs lookup from a shell
```bash
curl https://docs.kaelio.com/ktx/llms.txt
curl https://docs.kaelio.com/ktx/docs/getting-started/quickstart.md
```
## Guardrails
- Do not invent CLI flags. Fetch the relevant CLI reference page.
- Do not scrape rendered HTML when a `.md` route exists.
- Do not assume docs lookup requires agent-client configuration.
- Do not include credentials or secrets in prompts, URLs, or copied docs snippets.
- When docs and local CLI behavior disagree, prefer the local CLI output and mention the mismatch.

View file

@ -1,76 +0,0 @@
---
title: Markdown Access
description: Fetch ktx docs as llms.txt, llms-full.txt, or per-page Markdown.
---
**ktx** docs are available as plain Markdown so assistants do not need to parse the rendered HTML site.
## Index
Fetch the curated index:
```text
https://docs.kaelio.com/ktx/llms.txt
```
Use this file to discover high-value pages, task-specific entry points, and Markdown URLs.
## Full corpus
Fetch the complete docs corpus:
```text
https://docs.kaelio.com/ktx/llms-full.txt
```
Use this when an assistant needs broad context across setup, concepts, CLI reference, integrations, and troubleshooting. Prefer the smaller per-page Markdown route for narrow tasks.
## Per-page Markdown
Every docs page has a Markdown route:
```text
https://docs.kaelio.com/ktx/docs/getting-started/quickstart.md
https://docs.kaelio.com/ktx/docs/cli-reference/ktx-sl.md
https://docs.kaelio.com/ktx/docs/cli-reference/ktx-wiki.md
https://docs.kaelio.com/ktx/docs/guides/building-context.md
```
Requests that ask for Markdown can also use the normal docs URL with `Accept: text/markdown`:
```bash
curl -H "Accept: text/markdown" https://docs.kaelio.com/ktx/docs/getting-started/quickstart
```
## Recommended retrieval order
1. Fetch `/llms.txt`.
2. Select one or two relevant page Markdown URLs.
3. Fetch `/llms-full.txt` only when page-level docs are not enough.
## Output contract
Markdown responses are designed for agent consumption:
- Frontmatter is removed.
- Each page includes a title, description, canonical URL, and Markdown URL.
- Code blocks stay as code blocks.
- Tables stay as Markdown tables.
- Missing docs pages return a plain-text `404` instead of silently falling back to HTML.
## Page actions
Rendered docs pages include page-level actions near the title:
- **Copy MD** copies the generated Markdown for the current page.
- **View MD** opens the generated Markdown route.
- **Copy MDX** copies the source MDX for the current page.
## Common mistakes
| Mistake | Better path |
|---------|-------------|
| Scraping the HTML page for a docs answer | Fetch the `.md` route instead |
| Loading `/llms-full.txt` for a single CLI flag lookup | Fetch the relevant CLI reference page |
| Treating `/llms.txt` as complete documentation | Use it as an index, then fetch linked pages |
| Copying rendered text by hand | Use **Copy MD** or **Copy MDX** from the page actions |

View file

@ -1,10 +0,0 @@
{
"title": "AI Resources",
"defaultOpen": true,
"pages": [
"agent-quickstart",
"markdown-access",
"agent-instructions",
"prompt-recipes"
]
}

View file

@ -1,55 +0,0 @@
---
title: Prompt Recipes
description: Copyable prompts for common ktx agent workflows.
---
Use these prompts when asking a coding assistant to work with **ktx**. Replace project names, connection ids, and business terms with your own values.
## Learn the docs
```text
Read https://docs.kaelio.com/ktx/llms.txt first. Then fetch only the ktx Markdown pages needed for this task. Do not scrape rendered HTML unless no Markdown route exists.
```
## Set up a project
```text
Run npx skills add Kaelio/ktx --skill ktx and use the ktx skill to install
and configure ktx in this project.
```
## Find a command
```text
Find the correct ktx command for this task: <task>. Start with /llms.txt, then fetch the smallest relevant CLI reference .md page. Quote the exact command and flags from the docs.
```
## Explain setup
```text
Explain how to set up ktx for this repo. Read /docs/getting-started/quickstart.md and the relevant CLI reference pages. Summarize prerequisites, commands, generated files, and any credentials the user must provide manually.
```
## Compare concepts
```text
Explain the difference between these ktx concepts: <concepts>. Start from /llms.txt, fetch the relevant concept and guide pages as Markdown, and answer with links to the source pages.
```
## Review semantic changes
```text
Review the ktx semantic-layer and knowledge changes in this branch. Check that measures have clear definitions, joins use valid keys, hidden/internal columns are not exposed to agents, and validation passes. List concrete file and line issues first.
```
## Copy exact docs source
```text
Open the relevant ktx docs page and use the page action to copy the generated Markdown or source MDX. Preserve code fences and tables exactly.
```
## Update docs
```text
Update the ktx docs for agent readability. Keep AI Resources focused on docs consumption. After editing, verify /llms.txt, /llms-full.txt, and the affected .md routes.
```

View file

@ -48,6 +48,11 @@ directory. Use it from any directory to generate editor or agent schema files.
| `stop` | Stop the **ktx** daemon | | `stop` | Stop the **ktx** daemon |
| `status` | Show managed Python runtime status and readiness checks | | `status` | Show managed Python runtime status and readiness checks |
`install` is self-contained: **ktx** downloads its own pinned, checksum-verified
`uv` build under the runtime root and uses it to provision Python and the
runtime wheel. Nothing needs to be installed on `PATH` first; the host only
needs network access to `github.com` during the first install.
## `admin runtime` Options ## `admin runtime` Options
| Flag | Description | Default | | Flag | Description | Default |

View file

@ -0,0 +1,86 @@
---
title: "ktx completion"
description: "Print a shell completion script for tab completion."
---
Print a shell completion script for **ktx**. Once installed, pressing <kbd>Tab</kbd>
completes commands, subcommands, and flags, and - inside a **ktx** project - the
names of things that already exist: semantic-layer source names for
`ktx sl read` and `ktx sl validate`, wiki page keys for `ktx wiki read`, and
configured connection ids for `ktx connection test`, `ktx ingest`, and
`ktx sql`. This saves you from remembering exact source, page, or connection
names.
## Command signature
```bash
ktx completion <shell>
```
`<shell>` must be `zsh` or `bash`. The command writes the script to stdout; it
does not modify any files. Enable completion by evaluating the script in your
shell startup file.
## Installation
Add the matching line to your shell startup file, then restart your shell (or
`source` the file). `ktx` must be on your `PATH`.
```bash
# zsh — add to ~/.zshrc
eval "$(ktx completion zsh)"
```
```bash
# bash — add to ~/.bashrc
eval "$(ktx completion bash)"
```
To try it for the current session only, run the same `eval` line directly in
your terminal.
## What gets completed
| Position | Completions |
|----------|-------------|
| `ktx <Tab>` | Top-level commands (`setup`, `sl`, `wiki`, `ingest`, …) |
| `ktx sl <Tab>` | The `read` / `validate` / `query` subcommands |
| `ktx sl read <Tab>` | Existing semantic-layer source names |
| `ktx sl validate <Tab>` | Existing semantic-layer source names |
| `ktx wiki <Tab>` | The `read` subcommand |
| `ktx wiki read <Tab>` | Existing wiki page keys |
| `ktx connection test <Tab>` | Configured connection ids |
| `ktx ingest <Tab>` | Configured connection ids |
| `ktx sql --connection <Tab>` | Configured connection ids |
| `ktx completion <Tab>` | `zsh` or `bash` |
| `ktx <command> --<Tab>` | The command's flags and inherited global flags |
| `ktx sl --output <Tab>` | An option's allowed values (here `pretty`, `plain`, `json`) |
| `ktx sl --connection-id <Tab>` | Configured connection ids |
Source names, wiki page keys, and connection ids are read from the **ktx**
project resolved from your current directory (or `--project-dir` /
`KTX_PROJECT_DIR`). Outside a **ktx** project, completion still suggests
commands and flags but no project entities. Bare `ktx sl <Tab>` and
`ktx wiki <Tab>` complete subcommands instead of entity names because their
positional arguments are free-text search queries.
## Examples
```bash
# Print the zsh completion script
ktx completion zsh
# Print the bash completion script
ktx completion bash
# Install for zsh
echo 'eval "$(ktx completion zsh)"' >> ~/.zshrc
```
## Common errors
| Error | Cause | Recovery |
|-------|-------|----------|
| `error: command-argument value '<name>' is invalid for argument 'shell'. Allowed choices are zsh, bash.` | A shell other than `zsh` or `bash` was requested | Re-run with `ktx completion zsh` or `ktx completion bash` |
| Tab completion does nothing | The script was not evaluated, or `ktx` is not on `PATH` | Confirm the `eval` line is in your startup file, restart the shell, and verify `ktx --version` runs |
| Source, page, or connection names are missing | The current directory is not inside a **ktx** project | Run from the project directory, or pass `--project-dir`, or set `KTX_PROJECT_DIR` |

View file

@ -104,6 +104,6 @@ configured connection and exit non-zero if any probe fails.
| Error | Cause | Recovery | | Error | Cause | Recovery |
|-------|-------|----------| |-------|-------|----------|
| No connections configured | The project has no entries under `connections` | Run `ktx setup` and add a database or context-source connection | | No connections configured | The project has no entries under `connections` | Run `ktx setup` and add a database or context-source connection |
| Connection test fails | Credentials, network access, database, warehouse, or schema is invalid | Verify the same URL with the database's native client, then rerun `ktx setup` and reconfigure the connection | | Connection test fails | Credentials, network access, database, warehouse, or schema is invalid | Use the setup recovery menu to retry or re-enter details; if it still fails, verify the same URL with the database's native client |
| Mapping validation fails during setup | BI database mappings do not point at valid warehouse connections | Rerun `ktx setup` and update the context-source mapping selections | | Mapping validation fails during setup | BI database mappings do not point at valid warehouse connections | Use the setup recovery menu to retry validation or re-enter mapping selections; rerun `ktx setup` if you already exited |
| Notion page picker cannot run | The terminal is non-interactive or Notion discovery failed | Rerun interactive `ktx setup`, or use non-interactive setup flags with explicit root page ids | | Notion page picker cannot run | The terminal is non-interactive or Notion discovery failed | Rerun interactive `ktx setup`, or use non-interactive setup flags with explicit root page ids |

View file

@ -5,9 +5,11 @@ description: "Build or refresh ktx context, or capture text into ktx memory."
`ktx ingest` builds or refreshes **ktx** context from configured connections, and `ktx ingest` builds or refreshes **ktx** context from configured connections, and
can also capture free-form text into **ktx** memory. Database connections build can also capture free-form text into **ktx** memory. Database connections build
schema context. Context-source connections ingest metadata from tools such as enriched context — schema plus AI-generated descriptions, embeddings, and
dbt, Looker, Metabase, MetricFlow, LookML, and Notion. Pass `--text` or relationship evidence — and require a configured model and embeddings.
`--file` to capture inline text or text files into memory instead. Context-source connections ingest metadata from tools such as dbt, Looker,
Metabase, MetricFlow, LookML, and Notion. Pass `--text` or `--file` to capture
inline text or text files into memory instead.
## Command signature ## Command signature
@ -29,8 +31,6 @@ connection is selected.
| Flag | Description | Default | | Flag | Description | Default |
|------|-------------|---------| |------|-------------|---------|
| `--all` | Ingest all configured connections (same as bare invocation) | `false` | | `--all` | Ingest all configured connections (same as bare invocation) | `false` |
| `--fast` | Use deterministic fast database ingest | Stored connection default, or `fast` |
| `--deep` | Use deep database ingest with AI-generated descriptions, embeddings, and relationship evidence | Stored connection default, or `fast` |
| `--query-history` | Include database query-history usage patterns | Stored connection default | | `--query-history` | Include database query-history usage patterns | Stored connection default |
| `--no-query-history` | Skip database query-history usage patterns for this run | Stored connection default | | `--no-query-history` | Skip database query-history usage patterns for this run | Stored connection default |
| `--query-history-window-days <days>` | BigQuery/Snowflake query-history lookback window for this run | Stored connection default | | `--query-history-window-days <days>` | BigQuery/Snowflake query-history lookback window for this run | Stored connection default |
@ -44,12 +44,12 @@ connection is selected.
| `--yes` | Install required managed runtime features without prompting | `false` | | `--yes` | Install required managed runtime features without prompting | `false` |
| `--no-input` | Disable interactive terminal input | - | | `--no-input` | Disable interactive terminal input | - |
`--fast` and `--deep` are mutually exclusive. Depth flags apply only to Database ingest always builds enriched context and requires a configured model
database connections. Query-history flags apply only to database connections and embeddings (run `ktx setup`); connections without that configuration fail
before any work starts. Query-history flags apply only to database connections
that support query history. The window flag applies to BigQuery and Snowflake; that support query history. The window flag applies to BigQuery and Snowflake;
Postgres reads the current `pg_stat_statements` aggregate data instead of a Postgres reads the current `pg_stat_statements` aggregate data instead of a
time-windowed history table. Query-history ingest runs after fast ingest and time-windowed history table. Query-history ingest runs after the schema scan.
requires deep ingest readiness.
When more than one connection is selected, database ingest runs first, then When more than one connection is selected, database ingest runs first, then
context-source ingest and memory updates run for context-source connections. context-source ingest and memory updates run for context-source connections.
@ -72,14 +72,8 @@ ktx ingest
# Build one database or context-source connection # Build one database or context-source connection
ktx ingest warehouse ktx ingest warehouse
# Force deterministic fast database ingest
ktx ingest warehouse --fast
# Force deep database ingest with AI enrichment
ktx ingest warehouse --deep
# Include query-history usage patterns # Include query-history usage patterns
ktx ingest warehouse --deep --query-history ktx ingest warehouse --query-history
# Set the lookback window for BigQuery or Snowflake query history # Set the lookback window for BigQuery or Snowflake query history
ktx ingest warehouse --query-history-window-days 30 ktx ingest warehouse --query-history-window-days 30
@ -149,13 +143,51 @@ verbosity:
KTX_INGEST_TRACE_LEVEL=trace ktx ingest metabase KTX_INGEST_TRACE_LEVEL=trace ktx ingest metabase
``` ```
### Profiling a slow ingest
Each timed phase and work unit records a `durationMs` in the trace, and each
agent loop records its step count and token usage. To see where wall-clock time
went, enable profiling and **ktx** prints a rolled-up breakdown to stderr at the
end of the run. There are two ways to turn it on, and two output formats.
Turn it on per run with the `KTX_PROFILE_INGEST` environment variable, or
persistently with `ingest.profile` in `ktx.yaml` (useful for CI or while
iterating on a slow source):
```bash
KTX_PROFILE_INGEST=1 ktx ingest metabase # human-readable table
KTX_PROFILE_INGEST=json ktx ingest metabase # raw JSON for coding agents
```
```yaml
ingest:
profile: true # human table; use "json" for the machine-readable form
```
Both formats report total wall time, time per phase, and the slowest work units,
splitting each work unit's agent-loop time into model time versus tool-execution
time. The `json` form emits the full structured profile (raw milliseconds and
token counts, stable keys) plus a `summary.headline` one-line diagnosis, so a
coding agent can parse it directly instead of scraping the table. If both the env
var and the config request profiling, `json` wins. Example headline:
```text
Slowest phase: reconciliation (2m 05s, 48% of wall time). 2 work units (1 failed), ~88% model generation vs ~12% tools.
```
Work units run serially by default (`ingest.workUnits.maxConcurrency` is `1`);
raise it in `ktx.yaml` if the profile shows the run is bound by serialized
work-unit agent loops. If the provider reports an LLM rate limit, **ktx** shows
a transient wait message and temporarily reduces effective work-unit concurrency
according to `ingest.rateLimit`.
## Common errors ## Common errors
| Error | Cause | Recovery | | Error | Cause | Recovery |
|-------|-------|----------| |-------|-------|----------|
| Connection not configured | The connection id is not present in `ktx.yaml` | Add the connection with `ktx setup` or update `ktx.yaml` | | Connection not configured | The connection id is not present in `ktx.yaml` | Add the connection with `ktx setup` or update `ktx.yaml` |
| Deep readiness is missing | `--deep` or query history needs model, embedding, and scan-enrichment configuration | Run `ktx setup` or rerun with `--fast` | | Enrichment is not configured | Database ingest needs a model, embeddings, and scan-enrichment configuration | Run `ktx setup` to configure a model and embeddings |
| Query history is unsupported | The selected database driver does not support query history | Run fast ingest without query-history flags | | Query history is unsupported | The selected database driver does not support query history | Run ingest without query-history flags |
| Python runtime is missing | The selected ingest target needs runtime-backed SQL analysis or source parsing | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx admin runtime install` command | | Python runtime is missing | The selected ingest target needs runtime-backed SQL analysis or source parsing | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx admin runtime install` command |
| Context-source options were ignored | Depth and query-history flags were supplied for a context-source connection | Omit database-only flags when ingesting context-source connections | | Context-source options were ignored | Query-history flags were supplied for a context-source connection | Omit database-only flags when ingesting context-source connections |
| Text ingest stops early | `--fail-fast` was used and one item failed | Fix the failed item or rerun without `--fail-fast` to collect all failures | | Text ingest stops early | `--fail-fast` was used and one item failed | Fix the failed item or rerun without `--fail-fast` to collect all failures |

View file

@ -68,3 +68,4 @@ hosts and origins for browser clients.
| No **ktx** project found | Current directory has no `ktx.yaml` and `KTX_PROJECT_DIR` is unset | Run from a **ktx** project or pass `--project-dir <path>` | | No **ktx** project found | Current directory has no `ktx.yaml` and `KTX_PROJECT_DIR` is unset | Run from a **ktx** project or pass `--project-dir <path>` |
| Non-loopback host rejected | The server needs token auth before binding beyond localhost | Pass `--token <token>` or set `KTX_MCP_TOKEN` | | Non-loopback host rejected | The server needs token auth before binding beyond localhost | Pass `--token <token>` or set `KTX_MCP_TOKEN` |
| Client cannot connect | Host, port, token, allowed host, or allowed origin does not match the client | Check `ktx mcp status`, then restart with explicit `--host`, `--port`, `--allowed-host`, and `--allowed-origin` values | | Client cannot connect | Host, port, token, allowed host, or allowed origin does not match the client | Check `ktx mcp status`, then restart with explicit `--host`, `--port`, `--allowed-host`, and `--allowed-origin` values |
| A Python-backed tool reports a runtime install failure | A tool that needs the managed Python runtime (metric compute, query-history SQL analysis) ran on a host that cannot reach `github.com` to download the pinned `uv` and Python | The server still starts and serves catalog and search tools. Restore network access and retry, or pre-build the runtime where network is available: `ktx admin runtime install --yes` |

View file

@ -29,6 +29,7 @@ below.
| `--agents` | Install agent configuration and rules only | `false` | | `--agents` | Install agent configuration and rules only | `false` |
| `--target <target>` | Agent target: `claude-code`, `claude-desktop`, `codex`, `cursor`, `opencode`, or `universal` | - | | `--target <target>` | Agent target: `claude-code`, `claude-desktop`, `codex`, `cursor`, `opencode`, or `universal` | - |
| `--global` | Install agent integration into the global target scope for `claude-code` or `codex` | `false` | | `--global` | Install agent integration into the global target scope for `claude-code` or `codex` | `false` |
| `--install-dir <path>` | Directory to install project-scoped agent config into. Defaults to the ktx project directory; resolved against the current directory and created if missing. Use it to install `.claude/`, `.mcp.json`, and rules where you open your agent (e.g. `--install-dir .`). Mutually exclusive with `--global` and `--local` | ktx project dir |
| `--yes` | Accept project creation and runtime install defaults where setup asks for confirmation | `false` | | `--yes` | Accept project creation and runtime install defaults where setup asks for confirmation | `false` |
| `--no-input` | Disable interactive terminal input | - | | `--no-input` | Disable interactive terminal input | - |
@ -51,9 +52,9 @@ prompts.
| Flag | Description | | Flag | Description |
|------|-------------| |------|-------------|
| `--llm-backend <backend>` | LLM backend: `anthropic`, `vertex`, or `claude-code` | | `--llm-backend <backend>` | LLM backend: `anthropic`, `vertex`, `claude-code`, or `codex` |
| `--llm-backend claude-code` | Use the local Claude Code session for **ktx** LLM calls | | `--llm-backend claude-code` | Use the local Claude Code session for **ktx** LLM calls |
| `--llm-model <model>` | LLM model ID or backend model alias to validate and save | | `--llm-backend codex` | Use local Codex authentication for **ktx** LLM calls |
| `--anthropic-api-key-env <name>` | Environment variable containing the Anthropic API key | | `--anthropic-api-key-env <name>` | Environment variable containing the Anthropic API key |
| `--anthropic-api-key-file <path>` | File containing the Anthropic API key | | `--anthropic-api-key-file <path>` | File containing the Anthropic API key |
| `--vertex-project <project>` | Vertex AI project ID, `env:NAME`, or `file:/path` reference | | `--vertex-project <project>` | Vertex AI project ID, `env:NAME`, or `file:/path` reference |
@ -62,9 +63,17 @@ prompts.
Choose only one Anthropic credential source. Anthropic credential flags are only Choose only one Anthropic credential source. Anthropic credential flags are only
valid with the Anthropic backend; Vertex flags are only valid with the Vertex valid with the Anthropic backend; Vertex flags are only valid with the Vertex
backend. The `claude-code` backend uses local Claude Code authentication instead backend. The `claude-code` and `codex` backends use local authentication instead
of Anthropic API key or Vertex flags. For Claude Code, `--llm-model` accepts of Anthropic API key or Vertex flags. After you choose a backend, `ktx setup`
`sonnet`, `opus`, `haiku`, or a full Claude model ID. writes that backend's per-role model preset to `ktx.yaml`. To change a model,
edit the matching `llm.models.<role>` value in `ktx.yaml`.
With `--no-input`, `ktx setup` does not assume a default LLM provider, because
every backend needs credentials only you can supply. Pass `--llm-backend`
explicitly. Note that `--target` selects the agent integration, not the LLM
provider: `ktx setup --target claude-code --no-input` still needs
`--llm-backend claude-code` to use your Claude subscription for **ktx** LLM
calls.
### Embeddings ### Embeddings
@ -117,6 +126,14 @@ incomplete.
MySQL, and SQL Server; `schema_names` for Snowflake; `dataset_ids` for MySQL, and SQL Server; `schema_names` for Snowflake; `dataset_ids` for
BigQuery; and `databases` for ClickHouse. BigQuery; and `databases` for ClickHouse.
With `--no-input`, scope for a scope-bearing driver (PostgreSQL, MySQL,
ClickHouse, SQL Server, BigQuery, Snowflake) must come from `--database-schema`
or from existing connection config in `ktx.yaml` (for example
`connections.<id>.dataset_ids`). When neither is set, the database step fails
fast and prints the missing scope flag and config key — non-interactive setup
never auto-discovers and scans every schema. SQLite has no scope and is
unaffected.
### Query History ### Query History
| Flag | Description | | Flag | Description |
@ -131,8 +148,8 @@ BigQuery; and `databases` for ClickHouse.
Query history setup is supported for Postgres, BigQuery, and Snowflake. The Query history setup is supported for Postgres, BigQuery, and Snowflake. The
window flag applies to BigQuery and Snowflake; Postgres reads the current window flag applies to BigQuery and Snowflake; Postgres reads the current
`pg_stat_statements` aggregate data instead of a time-windowed history table. `pg_stat_statements` aggregate data instead of a time-windowed history table.
Enabling query history makes deep ingest readiness matter for later Later `ktx ingest` runs build enriched context and need a configured model and
`ktx ingest` runs. embeddings, including when query history is enabled.
When query history is enabled for PostgreSQL, Snowflake, or BigQuery, When query history is enabled for PostgreSQL, Snowflake, or BigQuery,
`ktx setup` runs a non-blocking readiness probe after the connection test `ktx setup` runs a non-blocking readiness probe after the connection test
@ -142,6 +159,13 @@ fix the prerequisite. If the later schema-context build also fails, interactive
setup offers **Disable query history and retry** so you can finish database setup offers **Disable query history and retry** so you can finish database
setup with `connections.<id>.context.queryHistory.enabled: false`. setup with `connections.<id>.context.queryHistory.enabled: false`.
After the schema scan completes, setup can derive query-history service-account
filters from in-scope history. If **ktx** finds clear operational roles, it
prints each proposed exclusion with a reason and writes
`connections.<id>.context.queryHistory.filters.serviceAccounts` only when you
apply the proposal. In non-interactive setup with `--yes`, the proposal is
applied automatically. Existing `serviceAccounts` blocks are never overwritten.
For BigQuery, the remediation tells you to grant `roles/bigquery.resourceViewer` For BigQuery, the remediation tells you to grant `roles/bigquery.resourceViewer`
on the BigQuery project, or grant a custom role that contains on the BigQuery project, or grant a custom role that contains
`bigquery.jobs.listAll`. `bigquery.jobs.listAll`.
@ -160,9 +184,9 @@ sources. This is equivalent to passing `--skip-sources` in scripted setup.
| `--source-git-url <url>` | Git URL for dbt, MetricFlow, or LookML | | `--source-git-url <url>` | Git URL for dbt, MetricFlow, or LookML |
| `--source-branch <branch>` | Git branch for context-source setup | | `--source-branch <branch>` | Git branch for context-source setup |
| `--source-subpath <path>` | Repo subpath for context-source setup | | `--source-subpath <path>` | Repo subpath for context-source setup |
| `--source-auth-token-ref <ref>` | `env:` or `file:` credential reference for source repo auth | | `--source-auth-token-ref <ref>` | `env:` or `file:` credential reference for source repo auth or Notion integration token |
| `--source-url <url>` | Source service URL for Metabase or Looker | | `--source-url <url>` | Source service URL for Metabase or Looker |
| `--source-api-key-ref <ref>` | `env:` or `file:` API key reference for Metabase or Notion | | `--source-api-key-ref <ref>` | `env:` or `file:` API key reference for Metabase |
| `--source-client-id <id>` | Looker client id | | `--source-client-id <id>` | Looker client id |
| `--source-client-secret-ref <ref>` | `env:` or `file:` Looker client secret reference | | `--source-client-secret-ref <ref>` | `env:` or `file:` Looker client secret reference |
| `--source-warehouse-connection-id <id>` | Warehouse connection id used for context-source mapping | | `--source-warehouse-connection-id <id>` | Warehouse connection id used for context-source mapping |
@ -185,12 +209,22 @@ ktx setup
# Run setup for a specific project directory # Run setup for a specific project directory
ktx setup --project-dir ./analytics ktx setup --project-dir ./analytics
# Use Claude Code with Opus for ktx LLM calls # Use Claude Code for ktx LLM calls
ktx setup \ ktx setup \
--project-dir ./analytics \ --project-dir ./analytics \
--llm-backend claude-code \ --llm-backend claude-code
--llm-model opus
# Configure **ktx** to use local Codex authentication for LLM work
ktx setup --llm-backend codex --no-input
```
When you choose `--llm-backend codex`, setup prints a warning if the public
Codex SDK and CLI surface cannot prove full Claude-Code-style isolation. The
backend restricts **ktx** runtime MCP tools to each run, but Codex may still
load user Codex config and built-in command execution or read-only file
capabilities.
```bash
# Script a Postgres connection that reads its URL from the environment # Script a Postgres connection that reads its URL from the environment
ktx setup \ ktx setup \
--project-dir ./analytics \ --project-dir ./analytics \
@ -221,6 +255,14 @@ ktx setup \
--source-warehouse-connection-id warehouse \ --source-warehouse-connection-id warehouse \
--metabase-database-id 1 --metabase-database-id 1
# Add a Notion source that crawls selected root pages
ktx setup \
--source notion \
--source-connection-id notion-main \
--source-auth-token-ref env:NOTION_TOKEN \
--notion-crawl-mode selected_roots \
--notion-root-page-id abc123def456
# Install project-scoped agent integration for Codex # Install project-scoped agent integration for Codex
ktx setup --agents --target codex ktx setup --agents --target codex
``` ```
@ -250,6 +292,7 @@ Use `ktx status` for repeatable readiness checks after setup exits.
|-------|-------|----------| |-------|-------|----------|
| Setup resumes an unexpected project | `KTX_PROJECT_DIR` or nearest `ktx.yaml` points to another directory | Pass `--project-dir <path>` explicitly | | Setup resumes an unexpected project | `KTX_PROJECT_DIR` or nearest `ktx.yaml` points to another directory | Pass `--project-dir <path>` explicitly |
| Setup cannot run in CI | Required values are missing and `--no-input` disables prompts | Provide the relevant automation flags or create a fixture `ktx.yaml` | | Setup cannot run in CI | Required values are missing and `--no-input` disables prompts | Provide the relevant automation flags or create a fixture `ktx.yaml` |
| `Missing LLM backend: pass --llm-backend …` | `--no-input` setup ran without an LLM backend; `--target` does not select one | Pass `--llm-backend claude-code`, `codex`, `anthropic`, or `vertex` (with that backend's credential flags) |
| Provider health check fails | Provider key, model id, Vertex project, or Vertex location is invalid | Fix the `env:` or `file:` reference and rerun setup | | Provider health check fails | Provider key, model id, Vertex project, or Vertex location is invalid | Fix the `env:` or `file:` reference and rerun setup |
| Python runtime is missing | The selected setup needs runtime-backed agent, query-history, Looker, or local embedding features | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx admin runtime install` command | | Python runtime is missing | The selected setup needs runtime-backed agent, query-history, Looker, or local embedding features | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx admin runtime install` command |
| `--enable-query-history` is rejected | The selected database driver does not support query history | Use Postgres, BigQuery, or Snowflake, or rerun without query-history flags | | `--enable-query-history` is rejected | The selected database driver does not support query history | Use Postgres, BigQuery, or Snowflake, or rerun without query-history flags |

View file

@ -11,13 +11,16 @@ the vocabulary agents use to generate correct SQL.
```bash ```bash
ktx sl [options] [query...] # list (bare) or search (with query) ktx sl [options] [query...] # list (bare) or search (with query)
ktx sl validate <sourceName> [options] ktx sl read <sourceName>
ktx sl validate <sourceName>
ktx sl query [options] ktx sl query [options]
``` ```
- Bare `ktx sl` lists semantic sources. - Bare `ktx sl` lists semantic sources.
- `ktx sl <query...>` searches semantic sources (multi-word queries are - `ktx sl <query...>` searches semantic sources. Multi-word queries are joined
joined with a space). with a space.
- `ktx sl read <sourceName>` prints the YAML for one source. Add
`--connection-id` only when the source name exists in multiple connections.
- `ktx sl validate` and `ktx sl query` remain as explicit subcommands. - `ktx sl validate` and `ktx sl query` remain as explicit subcommands.
## Subcommands ## Subcommands
@ -26,6 +29,7 @@ ktx sl query [options]
|-----------|-------------| |-----------|-------------|
| (none, no query) | List semantic sources | | (none, no query) | List semantic sources |
| (none, with query) | Search semantic sources | | (none, with query) | Search semantic sources |
| `read <sourceName>` | Print the YAML for one semantic source |
| `validate <sourceName>` | Validate a semantic source against the database schema | | `validate <sourceName>` | Validate a semantic source against the database schema |
| `query` | Compile or execute a semantic query | | `query` | Compile or execute a semantic query |
@ -40,17 +44,23 @@ ktx sl query [options]
| `--output <mode>` | Output mode: `pretty` (default in TTY), `plain` (TSV), or `json` | `pretty` | | `--output <mode>` | Output mode: `pretty` (default in TTY), `plain` (TSV), or `json` | `pretty` |
| `--json` | Shortcut for `--output=json` (overrides `--output`) | `false` | | `--json` | Shortcut for `--output=json` (overrides `--output`) | `false` |
### `sl read`
| Flag | Description | Default |
|------|-------------|---------|
| `--connection-id <id>` | Optional **ktx** connection id for disambiguation | - |
### `sl validate` ### `sl validate`
| Flag | Description | Default | | Flag | Description | Default |
|------|-------------|---------| |------|-------------|---------|
| `--connection-id <id>` | **ktx** connection id (required) | - | | `--connection-id <id>` | Optional **ktx** connection id for disambiguation | - |
### `sl query` ### `sl query`
| Flag | Description | Default | | Flag | Description | Default |
|------|-------------|---------| |------|-------------|---------|
| `--connection-id <id>` | **ktx** connection id | - | | `--connection-id <id>` | Required **ktx** connection id | - |
| `--query-file <path>` | JSON semantic query file | - | | `--query-file <path>` | JSON semantic query file | - |
| `--measure <measure>` | Measure to query; repeatable (at least one required) | - | | `--measure <measure>` | Measure to query; repeatable (at least one required) | - |
| `--dimension <dimension>` | Dimension to include; repeatable | - | | `--dimension <dimension>` | Dimension to include; repeatable | - |
@ -65,8 +75,9 @@ ktx sl query [options]
| `--no-input` | Disable interactive managed runtime installation | - | | `--no-input` | Disable interactive managed runtime installation | - |
| `--max-rows <n>` | Maximum rows to return when executing | - | | `--max-rows <n>` | Maximum rows to return when executing | - |
`sl query` requires at least one `--measure` unless `--query-file` is set. `sl query` requires `--connection-id` and at least one `--measure` unless
`--query-file` should point to a JSON semantic query object. `--query-file` is set. `--query-file` must point to a JSON semantic query
object.
## Examples ## Examples
@ -83,7 +94,16 @@ ktx sl --json
# Search sources as JSON # Search sources as JSON
ktx sl "revenue" --json ktx sl "revenue" --json
# Validate a source against the live schema # Print the YAML for a source name that is unique across connections
ktx sl read orders
# Print the YAML for a source name that exists in multiple connections
ktx sl --connection-id my-warehouse read orders
# Validate a source name that is unique across connections
ktx sl validate orders
# Validate a source name that exists in multiple connections
ktx sl validate orders --connection-id my-warehouse ktx sl validate orders --connection-id my-warehouse
# Compile a query and view the generated SQL # Compile a query and view the generated SQL
@ -144,6 +164,12 @@ shows `#1`, `#2`, and later rank badges for the displayed results. Plain and
JSON output keep the raw `score` value, which is a ranking score rather than a JSON output keep the raw `score` value, which is a ranking score rather than a
percentage. percentage.
`ktx sl read <sourceName>` prints the source YAML directly to stdout when the
source name is unique across connections. If the name exists in multiple
connections, rerun the command with `--connection-id <id>`. The command does
not wrap output in pretty, plain, or JSON formatting, so it can be piped to
other tools.
```json ```json
{ {
"sql": "SELECT orders.status, SUM(orders.total_amount) AS total_revenue FROM public.orders GROUP BY orders.status", "sql": "SELECT orders.status, SUM(orders.total_amount) AS total_revenue FROM public.orders GROUP BY orders.status",
@ -160,7 +186,8 @@ percentage.
| Error | Cause | Recovery | | Error | Cause | Recovery |
|-------|-------|----------| |-------|-------|----------|
| Source not found | Source name or connection id is wrong | Run `ktx sl --json` and retry with an exact source name and connection id | | Source not found | Source name or connection id is wrong | Run `ktx sl <query>` or `ktx sl --connection-id <id>` to find the exact source name, then retry `ktx sl read <sourceName>` or `ktx sl validate <sourceName>` |
| Source name is ambiguous | The same source name exists in multiple connections | Rerun with `--connection-id <id>` from the error message |
| Validation fails | YAML references missing columns, invalid joins, or invalid SQL expressions | Fix the source YAML and rerun `ktx sl validate` | | Validation fails | YAML references missing columns, invalid joins, or invalid SQL expressions | Fix the source YAML and rerun `ktx sl validate` |
| Query compile fails | Measure, dimension, filter, or segment name is invalid | Search sources with `ktx sl <query>`, inspect the source YAML in your project files, then retry using declared fields | | Query compile fails | Measure, dimension, filter, or segment name is invalid | Search sources with `ktx sl <query>`, inspect the source YAML in your project files, then retry using declared fields |
| Execution returns too many rows | `--max-rows` is missing or too high | Add `--max-rows` with a bounded value before executing | | Execution returns too many rows | `--max-rows` is missing or too high | Add `--max-rows` with a bounded value before executing |

View file

@ -21,7 +21,7 @@ ktx status [options]
| `--json` | Print JSON output | `false` | | `--json` | Print JSON output | `false` |
| `-v`, `--verbose` | Show every check, including passing ones | `false` | | `-v`, `--verbose` | Show every check, including passing ones | `false` |
| `--validate` | Only validate the `ktx.yaml` schema; skip readiness checks | `false` | | `--validate` | Only validate the `ktx.yaml` schema; skip readiness checks | `false` |
| `--fast` | Skip checks that require external communication (query-history readiness probes and Claude Code auth probe) | `false` | | `--fast` | Skip checks that require external communication (query-history readiness probes, Claude Code auth probe, and Codex auth probe) | `false` |
| `--no-input` | Disable interactive terminal input | - | | `--no-input` | Disable interactive terminal input | - |
## Examples ## Examples
@ -39,7 +39,7 @@ ktx status --verbose
# Validate ktx.yaml without running readiness checks # Validate ktx.yaml without running readiness checks
ktx status --validate ktx status --validate
# Skip slow probes (query-history readiness, Claude Code auth) # Skip slow probes (query-history readiness, Claude Code auth, Codex auth)
ktx status --fast ktx status --fast
# Check a project from another directory # Check a project from another directory
@ -57,6 +57,16 @@ flow, then rerun `ktx status`. Use `--fast` to skip this probe (useful in CI
or offline contexts); skipped checks render as `-` and carry or offline contexts); skipped checks render as `-` and carry
`"status": "skipped"` in JSON output. `"status": "skipped"` in JSON output.
For `llm.provider.backend: codex`, `ktx status` runs a minimal non-interactive
Codex request. If the probe fails, authenticate Codex locally with the Codex CLI
and verify the Codex CLI installation.
When `llm.provider.backend: codex` is configured, `ktx status` also prints a
warning when the installed public Codex SDK and CLI surface cannot prove full
Claude-Code-style isolation. The warning does not block authenticated Codex
usage, but it marks the project status as partial so you can make an explicit
runtime-isolation decision.
A `Local data` section summarises what the project has accumulated locally: A `Local data` section summarises what the project has accumulated locally:
ingest run counts, last completed timestamp per connection, knowledge page ingest run counts, last completed timestamp per connection, knowledge page
counts by scope, semantic-layer source and dictionary value counts, and the counts by scope, semantic-layer source and dictionary value counts, and the
@ -84,6 +94,6 @@ stats, and are always shown (they do not require external communication).
|-------|-------|----------| |-------|-------|----------|
| No **ktx** project found | Current directory has no `ktx.yaml` and `KTX_PROJECT_DIR` is unset | `ktx status` runs setup checks; run from a **ktx** project or set `KTX_PROJECT_DIR` for project checks | | No **ktx** project found | Current directory has no `ktx.yaml` and `KTX_PROJECT_DIR` is unset | `ktx status` runs setup checks; run from a **ktx** project or set `KTX_PROJECT_DIR` for project checks |
| Project config check fails | The project directory is missing or has an invalid `ktx.yaml` | Run `ktx setup` to resume setup | | Project config check fails | The project directory is missing or has an invalid `ktx.yaml` | Run `ktx setup` to resume setup |
| Schema validation fails | `ktx.yaml` does not match the current config schema | Run `ktx status --validate --json` for structured issue details, then edit `ktx.yaml` or rerun `ktx setup` | | Schema validation fails | A field **ktx** recognizes has an invalid value. Unrecognized keys are reported as non-blocking warnings (exit `0`), not failures | Run `ktx status --validate --json` for structured issue details, then edit `ktx.yaml` or rerun `ktx setup` |
| Semantic search check warns | Embeddings are not configured or the provider probe failed | Run `ktx setup` or inspect the check's `fix` field in JSON output | | Semantic search check warns | Embeddings are not configured or the provider probe failed | Run `ktx setup` or inspect the check's `fix` field in JSON output |
| Query history check warns | A database has query history enabled but the warehouse prerequisites are missing | Fix the warehouse extension, grants, or history access, then rerun `ktx status` | | Query history check warns | A database has query history enabled but the warehouse prerequisites are missing | Fix the warehouse extension, grants, or history access, then rerun `ktx status` |

View file

@ -1,21 +1,24 @@
--- ---
title: "ktx wiki" title: "ktx wiki"
description: "List or search wiki pages." description: "List, search, or read wiki pages."
--- ---
List and search wiki pages in your **ktx** project. Wiki pages are Markdown List, search, and read wiki pages in your **ktx** project. Wiki pages are
documents that capture business definitions, rules, and gotchas. Agents search Markdown documents that capture business definitions, rules, and gotchas.
them for context when answering questions about your data. Agents search them for context when answering questions about your data.
## Command signature ## Command signature
```bash ```bash
ktx wiki [options] [query...] ktx wiki [options] [query...] # list (bare) or search (with query)
ktx wiki read <key>
``` ```
- Bare `ktx wiki` lists local wiki pages. - Bare `ktx wiki` lists local wiki pages.
- `ktx wiki <query...>` searches local wiki pages (multi-word queries are - `ktx wiki <query...>` searches local wiki pages. Multi-word queries are
joined with a space). joined with a space.
- `ktx wiki read <key>` prints the whole Markdown file for one wiki page,
including YAML frontmatter.
Edit the Markdown files under `wiki/` directly, or ingest source content with Edit the Markdown files under `wiki/` directly, or ingest source content with
`ktx ingest`, when you need to add or update wiki knowledge. `ktx ingest`, when you need to add or update wiki knowledge.
@ -50,6 +53,9 @@ ktx wiki "monthly recurring revenue"
# Search wiki pages as JSON # Search wiki pages as JSON
ktx wiki "monthly recurring revenue" --json --limit 10 ktx wiki "monthly recurring revenue" --json --limit 10
# Print the exact Markdown file for a known page key
ktx wiki read revenue-definitions
# Print search results as TSV # Print search results as TSV
ktx wiki "monthly recurring revenue" --output plain ktx wiki "monthly recurring revenue" --output plain
@ -62,8 +68,10 @@ ktx --debug wiki "monthly recurring revenue" --json
Wiki commands print clack-style pretty output in a TTY and TSV-style plain Wiki commands print clack-style pretty output in a TTY and TSV-style plain
output when requested. JSON output wraps the items with a command metadata output when requested. JSON output wraps the items with a command metadata
envelope. Search results include `matchReasons` and `lanes` metadata so you can envelope. Search results include `matchReasons` and `lanes` metadata so you can
see whether lexical, token, or semantic search contributed to the ranking. Open see whether lexical, token, or semantic search contributed to the ranking. Use
the matching Markdown files directly when you need the full page contents. `ktx wiki read <key>` when you need the full page contents. Read output is the
exact Markdown file stored on disk, including YAML frontmatter, and is not
wrapped in pretty, plain, or JSON formatting.
Pretty search output shows `#1`, `#2`, and later rank badges for the displayed Pretty search output shows `#1`, `#2`, and later rank badges for the displayed
results. Plain and JSON output keep the raw `score` value, which is a ranking results. Plain and JSON output keep the raw `score` value, which is a ranking
score rather than a percentage. score rather than a percentage.
@ -121,4 +129,4 @@ stays machine-readable:
| Error | Cause | Recovery | | Error | Cause | Recovery |
|-------|-------|----------| |-------|-------|----------|
| Search returns no results | The query terms do not match summaries, tags, or content, and the semantic lane is unavailable or has no positive matches | Run with `--debug`, check the semantic lane status, retry with business synonyms, then create a page if the knowledge is missing | | Search returns no results | The query terms do not match summaries, tags, or content, and the semantic lane is unavailable or has no positive matches | Run with `--debug`, check the semantic lane status, retry with business synonyms, then create a page if the knowledge is missing |
| A page is missing | No Markdown file exists for that business context | Add a file under `wiki/` or run `ktx ingest <connectionId>` | | A page is missing | No Markdown file exists for that business context or `ktx wiki read <key>` used the wrong key | Run `ktx wiki <query>` to find the page key, then retry `ktx wiki read <key>` |

View file

@ -36,9 +36,11 @@ ktx
wiki wiki
list list
search <query> search <query>
read <key>
sl sl
list list
search <query> search <query>
read <sourceName>
validate <sourceName> validate <sourceName>
query query
sql sql
@ -57,6 +59,7 @@ ktx
stop stop
status status
reindex reindex
completion <shell>
``` ```
The public context-build entrypoint is `ktx ingest [connectionId]` or The public context-build entrypoint is `ktx ingest [connectionId]` or
@ -71,6 +74,56 @@ The public context-build entrypoint is `ktx ingest [connectionId]` or
| `-v`, `--version` | Show the CLI package name and version. | | `-v`, `--version` | Show the CLI package name and version. |
| `-h`, `--help` | Show help for the current command. | | `-h`, `--help` | Show help for the current command. |
## Update notices
> **Note:** The update notifier writes only to stderr and keeps command stdout
> unchanged.
When a newer package is available on your installed release channel, `ktx`
prints a short notice after the command finishes:
```text
↑ Update available: ktx 0.9.0 → 0.10.0
npm i -g @kaelio/ktx
```
Stable installs compare against the npm `latest` dist-tag.
Release-candidate installs compare against the `next` dist-tag and show:
```text
npm i -g @kaelio/ktx@next
```
The check is skipped for JSON output, CI, non-TTY stdout, and hidden completion
commands. To opt out explicitly, set any of these environment variables:
```bash
KTX_NO_UPDATE_CHECK=1
NO_UPDATE_NOTIFIER=1
DO_NOT_TRACK=1
```
The `ktx` CLI prints one npm command because globally installed binaries don't
expose a reliable runtime package-manager signal. If you prefer another global
package manager, use the equivalent command:
```bash
pnpm add -g @kaelio/ktx
yarn global add @kaelio/ktx
```
## Build-view star prompt
During an interactive context build, `ktx setup` and `ktx ingest` can show a dim
GitHub star reminder above the `Ctrl+C to stop` hint. **ktx** skips this prompt
for CI, non-TTY output, and `DO_NOT_TRACK=1`.
To suppress only this prompt while keeping other notices enabled, set:
```bash
KTX_NO_STAR=1
```
## Project resolution ## Project resolution
Most commands are project-aware. Pass `--project-dir <path>` when scripting or Most commands are project-aware. Pass `--project-dir <path>` when scripting or
@ -97,6 +150,10 @@ ktx ingest
ktx sl "revenue" ktx sl "revenue"
ktx wiki "revenue recognition" ktx wiki "revenue recognition"
# Print a known wiki page or semantic source
ktx wiki read revenue-definitions
ktx sl --connection-id warehouse read orders
# Execute read-only SQL # Execute read-only SQL
ktx sql --connection warehouse "select count(*) from public.orders" ktx sql --connection warehouse "select count(*) from public.orders"

View file

@ -11,6 +11,7 @@
"ktx-wiki", "ktx-wiki",
"ktx-status", "ktx-status",
"ktx-mcp", "ktx-mcp",
"ktx-admin" "ktx-admin",
"ktx-completion"
] ]
} }

View file

@ -0,0 +1,111 @@
---
title: AI Resources
description: How coding agents read, cite, and act on the ktx docs - Markdown endpoints, a task router, and copy-paste prompts.
---
This page is for coding assistants that read or cite the **ktx** docs. It covers
the machine-readable endpoints, a task router for common questions, and
copy-paste prompts. It is scoped to documentation lookup and safe command
discovery - to wire **ktx** into an agent client, see
[Agent Clients](/docs/integrations/agent-clients).
## Markdown endpoints
**ktx** docs are available as plain Markdown so assistants never have to parse
the rendered HTML site.
- [`/llms.txt`](/llms.txt) - a curated index of high-value pages and agent entry
points. **Start here.**
- [`/llms-full.txt`](/llms-full.txt) - the entire docs corpus in one response.
Use only when a task needs broad context across many pages.
- **Per-page Markdown** - append `.md` to any docs URL:
```text
https://docs.kaelio.com/ktx/docs/getting-started/quickstart.md
https://docs.kaelio.com/ktx/docs/cli-reference/ktx-sl.md
https://docs.kaelio.com/ktx/docs/guides/building-context.md
```
A request for any docs URL with an `Accept: text/markdown` header returns the
same Markdown without the `.md` suffix:
```bash
curl -H "Accept: text/markdown" https://docs.kaelio.com/ktx/docs/getting-started/quickstart
```
Each Markdown response leads with the page title, description, canonical URL, and
Markdown URL; frontmatter is stripped; code blocks and tables are preserved; and
missing pages return a plain-text `404` instead of falling back to HTML. Rendered
pages also expose a **Copy as Markdown** action near the title.
### Retrieval order
1. Fetch [`/llms.txt`](/llms.txt).
2. Pick one or two relevant per-page `.md` URLs.
3. Fetch [`/llms-full.txt`](/llms-full.txt) only when page-level docs are not
enough.
```bash
curl https://docs.kaelio.com/ktx/llms.txt
curl https://docs.kaelio.com/ktx/docs/getting-started/quickstart.md
```
## Task router
| User asks the agent to explain... | Read first | Then read |
|------------------------------------|------------|-----------|
| What **ktx** does | [Introduction](/docs/getting-started/introduction) | [The Context Layer](/docs/concepts/the-context-layer) |
| How to start from a checkout | [Quickstart](/docs/getting-started/quickstart) | [ktx setup](/docs/cli-reference/ktx-setup) |
| How to check project readiness | [ktx status](/docs/cli-reference/ktx-status) | [Quickstart](/docs/getting-started/quickstart) |
| How context gets built | [Building Context](/docs/guides/building-context) | [ktx ingest](/docs/cli-reference/ktx-ingest) |
| How semantic YAML works | [Writing Context](/docs/guides/writing-context) | [ktx sl](/docs/cli-reference/ktx-sl) |
| How machine-readable CLI output is shaped | [ktx sl](/docs/cli-reference/ktx-sl) | [ktx wiki](/docs/cli-reference/ktx-wiki) |
## Agent instructions
Paste this into a project or system prompt when an assistant needs to answer
from the **ktx** docs:
```text
When answering ktx docs questions:
1. Start with https://docs.kaelio.com/ktx/llms.txt.
2. Fetch the smallest relevant Markdown page (append .md to its docs URL).
3. Prefer the .md route over rendered HTML.
4. Use https://docs.kaelio.com/ktx/llms-full.txt only when the task needs broad docs context.
5. Quote commands exactly from docs pages.
6. If docs and local CLI behavior disagree, say what differs and prefer local verified output.
```
## Prompts
Replace project names, connection ids, and business terms with your own values.
**Install and configure ktx in a project**
```text
Run npx skills add Kaelio/ktx --skill ktx and use the ktx skill to install and configure ktx
```
**Find the right command**
```text
Find the correct ktx command for this task: <task>. Start with /llms.txt, then fetch the smallest relevant CLI reference .md page. Quote the exact command and flags from the docs.
```
**Review semantic changes**
```text
Review the ktx semantic-layer and wiki changes in this branch. Check that measures have clear definitions, joins use valid keys, hidden or internal columns are not exposed to agents, and validation passes. List concrete file and line issues first.
```
## Guardrails
- Do not invent CLI flags - fetch the relevant CLI reference page.
- Do not scrape rendered HTML when a `.md` route exists.
- Do not treat `/llms.txt` as complete documentation - use it as an index, then
fetch the linked pages.
- Do not include credentials or secrets in prompts, URLs, or copied docs
snippets.
- When docs and local CLI behavior disagree, prefer the local CLI output and
mention the mismatch.

View file

@ -1,5 +1,5 @@
{ {
"title": "Community", "title": "Community & Resources",
"defaultOpen": true, "defaultOpen": true,
"pages": ["support", "contributing", "telemetry"] "pages": ["support", "contributing", "telemetry", "ai-resources"]
} }

View file

@ -1,12 +1,15 @@
--- ---
title: Telemetry title: Telemetry
description: Understand what anonymous usage telemetry ktx collects and how to opt out. description: Understand what usage telemetry ktx collects and how to opt out.
--- ---
**ktx** collects anonymous, aggregated usage telemetry from interactive CLI **ktx** collects aggregated usage telemetry so maintainers can see
runs so maintainers can see which commands work, where setup fails, and which which commands work, where setup fails, and which parts of the data-agent
parts of the data-agent workflow need improvement. Telemetry is opt-out and workflow need improvement. Telemetry is opt-out: it turns on the first time you
disabled automatically in CI and non-interactive runs. run **ktx** in any way — an interactive command, a script, or an
agent-launched MCP server — and prints a one-time notice (to the terminal when
there is one, otherwise to standard error). It stays disabled in CI and whenever
an opt-out is set.
## Opt out ## Opt out
@ -17,23 +20,58 @@ Use any of these mechanisms to disable telemetry:
| `export KTX_TELEMETRY_DISABLED=1` | Disables telemetry for the shell and child processes | | `export KTX_TELEMETRY_DISABLED=1` | Disables telemetry for the shell and child processes |
| `export DO_NOT_TRACK=1` | Standard do-not-track environment variable | | `export DO_NOT_TRACK=1` | Standard do-not-track environment variable |
| `CI=1` | Automatic in CI | | `CI=1` | Automatic in CI |
| Non-TTY output | Automatic for pipes and scripts | | Edit `~/.ktx/telemetry.json` and set `"enabled": false` | Persistent for the machine, including the MCP server |
| Edit `~/.ktx/telemetry.json` and set `"enabled": false` | Persistent for the machine |
## What we collect ## What we collect
High-level signals only: which commands run, how long they take, whether they High-level signals: which commands run, how long they take, whether they
succeed or fail, and basic environment metadata (CLI version, Node version, OS succeed or fail, and basic environment metadata (CLI version, Node version, OS
platform). For project-level analysis, **ktx** sends a salted hash of the platform). When an operation fails, we also include diagnostic detail about the
project directory — never the raw path. error so we can debug it. For project-level analysis, **ktx** sends a salted
hash of the project directory to group events.
When an agent reaches **ktx** through MCP, we also record the connecting client
tool's self-reported name and version (for example Claude Desktop, Cursor, or
Cline) so we can see which agents people use **ktx** with. That describes the
tool, never you or your data.
## What we never collect ## What we never collect
- File paths, hostnames, environment variable values, or command arguments We build telemetry around counts and coarse signals, not the contents of your
- `ktx.yaml` contents, connection passwords, API keys, or tokens data or configuration. We don't deliberately collect your `ktx.yaml`, query
- Schema names, table names, column names, SQL text, or query results results, passwords, API keys, or access tokens.
- Error messages or stack traces
- Git remote URLs, Git user email, OS user, or hostname The one place environment-specific text can appear is failure diagnostics: when
an operation errors, the detail we record is the error as your tools reported
it, which can include identifiers from your setup. If you'd rather send nothing
at all, turn telemetry off using any of the options above.
## Error reports
When telemetry is enabled, **ktx** sends PostHog Error Tracking `$exception`
events for CLI and daemon exceptions. Error reports help group crashes and
handled failures into PostHog issues.
Error reports can include:
- Stack frames, including function names, local file paths, line numbers, and
SDK-provided source context.
- Error class names and raw error messages.
- Cause chains when the runtime exposes them.
- `source`, `handled`, and `fatal` diagnostic fields.
- Runtime version, OS, architecture, and CI fields.
- The hashed `projectId` when **ktx** knows the project.
Error reports never intentionally include:
- Secrets, credentials, API keys, tokens, cookies, signed URLs, or auth headers.
- Database URLs, connection strings, DSNs, raw argv, or raw environment values.
- SQL text, schema names, table names, or column names as explicit payload
properties.
- Customer row data.
- User prompt text or raw MCP arguments.
The same opt-out controls listed above disable error reports.
## Storage and retention ## Storage and retention

View file

@ -46,7 +46,7 @@ read, how to think, and where to put the results.
</p> </p>
<ul className="mt-3 space-y-2 text-sm leading-6 text-fd-foreground"> <ul className="mt-3 space-y-2 text-sm leading-6 text-fd-foreground">
<li><code className="text-[13px] font-semibold">llm</code> - provider, models, prompt cache</li> <li><code className="text-[13px] font-semibold">llm</code> - provider, models, prompt cache</li>
<li><code className="text-[13px] font-semibold">ingest</code> - adapters, embeddings, work units</li> <li><code className="text-[13px] font-semibold">ingest</code> - connectors, embeddings, work units</li>
<li><code className="text-[13px] font-semibold">scan</code> - enrichment, relationships</li> <li><code className="text-[13px] font-semibold">scan</code> - enrichment, relationships</li>
<li><code className="text-[13px] font-semibold">agent</code> - research-agent feature flags</li> <li><code className="text-[13px] font-semibold">agent</code> - research-agent feature flags</li>
</ul> </ul>
@ -66,8 +66,9 @@ read, how to think, and where to put the results.
## Minimal config ## Minimal config
A working `ktx.yaml` needs one entry in `connections`. Everything else accepts A working `ktx.yaml` needs one entry in `connections`. Everything else accepts
defaults. The example below is enough for `ktx ingest warehouse` to run a fast defaults. The example below registers a local Postgres connection; building
schema scan against a local Postgres. context with `ktx ingest warehouse` also needs a model and embeddings, which
`ktx setup` configures.
```yaml ```yaml
connections: connections:
@ -123,7 +124,7 @@ context-source drivers share the map.
Warehouse connections are open objects: the listed fields are validated, and Warehouse connections are open objects: the listed fields are validated, and
any other field is preserved and passed through to the connector. Use any other field is preserved and passed through to the connector. Use
`enabled_tables` to scope deep ingest to a specific list of `enabled_tables` to scope ingest to a specific list of
`schema.table` names - useful for smoke tests. `schema.table` names - useful for smoke tests.
```yaml ```yaml
@ -178,9 +179,22 @@ connections:
context: context:
queryHistory: queryHistory:
enabled: true enabled: true
enabledSchemas:
- orbit_raw
- orbit_analytics
minExecutions: 5 minExecutions: 5
``` ```
- `enabledSchemas`: Optional list of schema or dataset names that query-history
ingest may mine. Omit it to let **ktx** derive the modeled schema floor from
the connection and semantic-layer sources. Use `["*"]` to disable the floor
for discovery runs.
- `filters.serviceAccounts`: Optional service-account filter block. During
setup, when query history is enabled and no service-account block already
exists, **ktx** can propose exact role patterns such as `^svc_loader$` from
observed in-scope query history. The block uses `mode: exclude` and remains
hand-editable.
### Metabase ### Metabase
```yaml ```yaml
@ -330,15 +344,14 @@ setup:
## `storage` ## `storage`
`storage` controls where **ktx** keeps its own state and search index, and how `storage` controls where **ktx** keeps its own state and search index. Defaults
state changes are committed. Defaults work for a single-user local project. work for a single-user local project.
```yaml ```yaml
storage: storage:
state: sqlite # sqlite | postgres state: sqlite # sqlite | postgres
search: sqlite-fts5 # sqlite-fts5 | postgres-hybrid search: sqlite-fts5 # sqlite-fts5 | postgres-hybrid
git: git:
auto_commit: true
author: "ktx <ktx@example.com>" author: "ktx <ktx@example.com>"
``` ```
@ -346,8 +359,7 @@ storage:
|-------|------|---------|---------| |-------|------|---------|---------|
| `state` | `sqlite` \| `postgres` | `sqlite` | Backend for ktx state. `sqlite` uses `.ktx/db.sqlite`; `postgres` expects a configured Postgres connection. | | `state` | `sqlite` \| `postgres` | `sqlite` | Backend for ktx state. `sqlite` uses `.ktx/db.sqlite`; `postgres` expects a configured Postgres connection. |
| `search` | `sqlite-fts5` \| `postgres-hybrid` | `sqlite-fts5` | Backend for search indexes. `postgres-hybrid` combines lexical and vector search in Postgres. | | `search` | `sqlite-fts5` \| `postgres-hybrid` | `sqlite-fts5` | Backend for search indexes. `postgres-hybrid` combines lexical and vector search in Postgres. |
| `git.auto_commit` | `boolean` | `true` | When `true`, ktx auto-commits changes to the git-backed state store. | | `git.author` | `string` | `ktx <ktx@example.com>` | Git author identity for commits. Standard `Name <email>` form. |
| `git.author` | `string` | `ktx <ktx@example.com>` | Git author identity for auto-commits. Standard `Name <email>` form. |
## `llm` ## `llm`
@ -363,6 +375,10 @@ llm:
models: models:
default: claude-sonnet-4-6 default: claude-sonnet-4-6
triage: claude-haiku-4-5 triage: claude-haiku-4-5
candidateExtraction: claude-sonnet-4-6
curator: claude-opus-4-7
reconcile: claude-opus-4-7
repair: claude-haiku-4-5
promptCaching: promptCaching:
enabled: true enabled: true
systemTtl: 1h systemTtl: 1h
@ -375,13 +391,28 @@ llm:
| Field | Type | Default | Purpose | | Field | Type | Default | Purpose |
|-------|------|---------|---------| |-------|------|---------|---------|
| `provider.backend` | `none` \| `anthropic` \| `vertex` \| `gateway` \| `claude-code` | `none` | Selected backend. `none` disables LLM features. `claude-code` uses the local Claude Code session and needs no API key. | | `provider.backend` | `none` \| `anthropic` \| `vertex` \| `gateway` \| `claude-code` \| `codex` | `none` | Selected backend. `none` disables LLM features. `claude-code` uses the local Claude Code session and needs no API key. `codex` uses local Codex authentication and needs no API key. |
| `provider.anthropic.api_key` | `string` | - | Anthropic API key. Required when `backend: anthropic`. Accepts `env:` or `file:` references. | | `provider.anthropic.api_key` | `string` | - | Anthropic API key. Required when `backend: anthropic`. Accepts `env:` or `file:` references. |
| `provider.anthropic.base_url` | `string` | - | Override the Anthropic API base URL (proxy, self-hosted gateway). | | `provider.anthropic.base_url` | `string` | - | Override the Anthropic API base URL (proxy, self-hosted gateway). |
| `provider.gateway.api_key` / `base_url` | `string` | - | Credentials for an AI Gateway provider. Required when `backend: gateway`. | | `provider.gateway.api_key` / `base_url` | `string` | - | Credentials for an AI Gateway provider. Required when `backend: gateway`. |
| `provider.vertex.project` | `string` | - | Google Cloud project ID hosting the Vertex AI endpoint. | | `provider.vertex.project` | `string` | - | Google Cloud project ID hosting the Vertex AI endpoint. |
| `provider.vertex.location` | `string` | - | Vertex AI region (for example `us-east5`). Required when the `vertex` block is present. | | `provider.vertex.location` | `string` | - | Vertex AI region (for example `us-east5`). Required when the `vertex` block is present. |
Use `codex` when local Codex authentication should power **ktx** LLM work:
```yaml
llm:
provider:
backend: codex
models:
default: gpt-5.5
triage: gpt-5.5
candidateExtraction: gpt-5.5
curator: gpt-5.5
reconcile: gpt-5.5
repair: gpt-5.5
```
### Model roles ### Model roles
`models` overrides the per-role model. Keys are fixed; values are `models` overrides the per-role model. Keys are fixed; values are
@ -409,7 +440,7 @@ provider-specific model identifiers.
## `ingest` ## `ingest`
`ingest` controls how **ktx** builds context from your stack. It lists the `ingest` controls how **ktx** builds context from your stack. It lists the
adapters to run, the embedding provider used when adapters embed documents, connectors to run, the embedding provider used when connectors embed documents,
and the concurrency and failure policy for work units. and the concurrency and failure policy for work units.
```yaml ```yaml
@ -428,14 +459,24 @@ ingest:
stepBudget: 40 stepBudget: 40
maxConcurrency: 2 maxConcurrency: 2
failureMode: continue failureMode: continue
rateLimit:
enabled: true
throttleThreshold: 0.8
minConcurrencyUnderPressure: 1
maxWaitMs: 600000
retry:
maxAttempts: 6
baseDelayMs: 1000
maxDelayMs: 60000
jitter: true
``` ```
### Adapters ### Connectors
`adapters` is a list of adapter IDs that should run. Each ID matches a `adapters` is a list of connector IDs that should run. Each ID matches a
connector that **ktx** ships locally: connector that **ktx** ships locally:
| Adapter ID | What it ingests | | Connector ID | What it ingests |
|------------|-----------------| |------------|-----------------|
| `live-database` | Live warehouse introspection (schemas, tables, columns, samples). | | `live-database` | Live warehouse introspection (schemas, tables, columns, samples). |
| `historic-sql` | Query history from Postgres `pg_stat_statements`, BigQuery `INFORMATION_SCHEMA.JOBS`, or Snowflake query history. | | `historic-sql` | Query history from Postgres `pg_stat_statements`, BigQuery `INFORMATION_SCHEMA.JOBS`, or Snowflake query history. |
@ -445,7 +486,7 @@ connector that **ktx** ships locally:
| `looker` | Looker dashboards and looks via the API. | | `looker` | Looker dashboards and looks via the API. |
| `metabase` | Metabase cards, dashboards, and database mappings. | | `metabase` | Metabase cards, dashboards, and database mappings. |
| `notion` | Notion pages and databases for wiki context. | | `notion` | Notion pages and databases for wiki context. |
| `fake` | Test/demo adapter. Useful in fixtures. | | `fake` | Test/demo connector. Useful in fixtures. |
### Embeddings ### Embeddings
@ -474,6 +515,24 @@ handles failures.
| `workUnits.maxConcurrency` | `int > 0` | `1` | How many work units run in parallel. | | `workUnits.maxConcurrency` | `int > 0` | `1` | How many work units run in parallel. |
| `workUnits.failureMode` | `abort` \| `continue` | `continue` | `abort` stops the whole ingest run on the first failure; `continue` records it and keeps going. | | `workUnits.failureMode` | `abort` \| `continue` | `continue` | `abort` stops the whole ingest run on the first failure; `continue` records it and keeps going. |
### Rate limits
`rateLimit` controls provider-neutral pacing for LLM calls during ingest. When a
provider reports a subscription window, retry-after delay, or HTTP 429,
**ktx** pauses new work-unit model calls, shows a transient wait in the CLI,
and reduces work-unit concurrency while the provider is under pressure.
| Field | Type | Default | Purpose |
|-------|------|---------|---------|
| `rateLimit.enabled` | `boolean` | `true` | Master switch for ingest LLM rate-limit pacing and visible waits. |
| `rateLimit.throttleThreshold` | `number between 0 and 1` | `0.8` | Fraction of a known provider window at which **ktx** starts reducing concurrency. |
| `rateLimit.minConcurrencyUnderPressure` | `int > 0` | `1` | Effective work-unit concurrency while a provider is under rate-limit pressure. |
| `rateLimit.maxWaitMs` | `int > 0` | unset | Caps how long a single provider-reset wait can last. This bounds each wait, not the whole run: after a capped wait elapses **ktx** retries and may pause again. Omit to wait until the provider's reset time. |
| `rateLimit.retry.maxAttempts` | `int > 0` | `6` | Maximum attempts for a single rate-limited LLM call before the failure surfaces (counts the first try). Also bounds how far opaque backoff grows for responses without a reset time or retry-after value. |
| `rateLimit.retry.baseDelayMs` | `int > 0` | `1000` | Initial opaque retry delay in milliseconds. |
| `rateLimit.retry.maxDelayMs` | `int > 0` | `60000` | Maximum opaque retry delay in milliseconds. |
| `rateLimit.retry.jitter` | `boolean` | `true` | Add jitter to opaque retry delays. |
## `scan` ## `scan`
`scan` configures how schema-level inputs become structured context: `scan` configures how schema-level inputs become structured context:
@ -547,19 +606,6 @@ agent:
| `run_research.max_iterations` | `int ≥ 0` | `20` | Maximum tool-call iterations per research run. | | `run_research.max_iterations` | `int ≥ 0` | `20` | Maximum tool-call iterations per research run. |
| `run_research.default_toolset` | `string[]` | `[sl_query, wiki_search, sl_read_source]` | Tool identifiers exposed to the research agent. | | `run_research.default_toolset` | `string[]` | `[sl_query, wiki_search, sl_read_source]` | Tool identifiers exposed to the research agent. |
## `memory`
`memory` controls the agent memory subsystem.
```yaml
memory:
auto_commit: true
```
| Field | Type | Default | Purpose |
|-------|------|---------|---------|
| `auto_commit` | `boolean` | `true` | When `true`, ktx auto-commits memory updates to the git-backed store. |
## A full example ## A full example
Combining the blocks above: Combining the blocks above:
@ -584,13 +630,17 @@ storage:
state: sqlite state: sqlite
search: sqlite-fts5 search: sqlite-fts5
git: git:
auto_commit: true
author: "ktx <ktx@example.com>" author: "ktx <ktx@example.com>"
llm: llm:
provider: provider:
backend: claude-code backend: claude-code
models: models:
default: sonnet default: sonnet
triage: haiku
candidateExtraction: sonnet
curator: opus
reconcile: opus
repair: haiku
ingest: ingest:
adapters: adapters:
- live-database - live-database
@ -612,17 +662,25 @@ scan:
agent: agent:
run_research: run_research:
enabled: true enabled: true
memory:
auto_commit: true
``` ```
## Validating your config ## Validating your config
**ktx** validates `ktx.yaml` strictly: unknown keys at the top level or inside **ktx** validates `ktx.yaml` when it loads, and treats two kinds of problems
strict blocks cause setup and CLI commands to fail with a precise path differently:
(`scan.relationships.acceptThreshhold: Unrecognized key`). Warehouse
connections accept extra driver-specific fields, so passthrough values like - **An invalid value on a field ktx recognizes** (for example
`historicSql` and `context.queryHistory` are allowed. `llm.provider.backend: nope`) is a hard error. Setup and CLI commands stop and
report the exact path so you can fix it.
- **An unrecognized key** — one left over from a different **ktx** version, or a
typo such as `scan.relationships.acceptThreshhold` — is tolerated, not fatal.
**ktx** ignores the key and keeps running, so a misspelled field quietly falls
back to its default instead of taking effect. `ktx status` lists each ignored
key as a warning (and exits `0`) so you can remove or correct it when
convenient.
Warehouse connections accept extra driver-specific fields, so passthrough values
like `historicSql` and `context.queryHistory` are allowed.
To re-validate without running anything else: To re-validate without running anything else:

View file

@ -4,6 +4,7 @@ description: ktx is an open-source, self-improving context layer for data agents
--- ---
import { ProductMechanics } from "@/components/product-mechanics"; import { ProductMechanics } from "@/components/product-mechanics";
import { ProductRuntime } from "@/components/product-runtime";
<div className="not-prose mb-10"> <div className="not-prose mb-10">
<div> <div>
@ -23,7 +24,7 @@ import { ProductMechanics } from "@/components/product-mechanics";
> >
Make analytics context usable by agents Make analytics context usable by agents
</h1> </h1>
<p className="mt-4 max-w-2xl text-lg text-fd-muted-foreground" style={{ lineHeight: '1.7' }}> <p className="mt-4 max-w-full text-lg text-fd-muted-foreground" style={{ lineHeight: '1.7' }}>
{'ktx is an open-source context layer for data agents. It turns warehouse metadata, BI tool definitions, query history, docs, and approved metric definitions into reviewable files agents can search and execute.'} {'ktx is an open-source context layer for data agents. It turns warehouse metadata, BI tool definitions, query history, docs, and approved metric definitions into reviewable files agents can search and execute.'}
</p> </p>
</div> </div>
@ -59,6 +60,8 @@ serves that context to agents at runtime.
<ProductMechanics /> <ProductMechanics />
<ProductRuntime />
## Use it for ## Use it for
Use **ktx** when agents need more than raw database access. Agents can search wiki Use **ktx** when agents need more than raw database access. Agents can search wiki
@ -92,8 +95,8 @@ best first step for users; contributor setup lives in the community docs.
<Card title="CLI Reference" href="/docs/cli-reference/ktx"> <Card title="CLI Reference" href="/docs/cli-reference/ktx">
Complete flag and subcommand reference for every **ktx** command. Complete flag and subcommand reference for every **ktx** command.
</Card> </Card>
<Card title="Agent Quickstart" href="/docs/ai-resources/agent-quickstart"> <Card title="AI Resources" href="/docs/community/ai-resources">
Machine-readable docs and agent-facing setup notes. Machine-readable docs, a task router, and copy-paste agent prompts.
</Card> </Card>
</Cards> </Cards>

View file

@ -9,8 +9,8 @@ This guide takes a local analytics project from empty to agent-ready. You'll
install the CLI, run one guided setup command, and hand the context to a install the CLI, run one guided setup command, and hand the context to a
coding assistant. coding assistant.
If you're a coding assistant choosing a docs route, start with the If you're a coding assistant choosing a docs route, start with
[Agent Quickstart](/docs/ai-resources/agent-quickstart) instead. [AI Resources](/docs/community/ai-resources) instead.
<div <div
className="not-prose my-8 overflow-hidden rounded-2xl border" className="not-prose my-8 overflow-hidden rounded-2xl border"
@ -30,17 +30,18 @@ If you're a coding assistant choosing a docs route, start with the
</div> </div>
<div className="mt-2.5 text-base leading-relaxed text-fd-foreground"> <div className="mt-2.5 text-base leading-relaxed text-fd-foreground">
Try **ktx** against a real data stack - Postgres, dbt, Metabase, and Notion Try **ktx** against a real data stack - Postgres, dbt, Metabase, and Notion
pre-loaded with the Orbit demo corpus. The page lists demo credentials pre-loaded with the Orbit demo corpus. Hit **copy agent setup** on the page
you can paste straight into `ktx setup`. for a one-shot prompt that has an agent install the full four-source demo,
or grab the raw credentials to paste into `ktx setup` yourself.
</div> </div>
<a <a
href="https://kaelio.com/start" href="https://www.kaelio.com/start"
className="group mt-5 inline-flex items-center gap-2 rounded-full px-4 py-2.5 text-sm font-semibold text-white no-underline shadow-[inset_0_1px_0_rgba(255,255,255,0.35),0_2px_4px_rgba(255,138,77,0.2),0_10px_24px_-8px_rgba(255,138,77,0.55)] transition-all duration-200 hover:-translate-y-0.5 hover:shadow-[inset_0_1px_0_rgba(255,255,255,0.4),0_3px_6px_rgba(255,138,77,0.28),0_16px_30px_-8px_rgba(255,138,77,0.65)]" className="group mt-5 inline-flex items-center gap-2 rounded-full px-4 py-2.5 text-sm font-semibold text-white no-underline shadow-[inset_0_1px_0_rgba(255,255,255,0.35),0_2px_4px_rgba(255,138,77,0.2),0_10px_24px_-8px_rgba(255,138,77,0.55)] transition-all duration-200 hover:-translate-y-0.5 hover:shadow-[inset_0_1px_0_rgba(255,255,255,0.4),0_3px_6px_rgba(255,138,77,0.28),0_16px_30px_-8px_rgba(255,138,77,0.65)]"
style={{ style={{
background: 'linear-gradient(180deg, #ff9d63 0%, #f97316 100%)', background: 'linear-gradient(180deg, #ff9d63 0%, #f97316 100%)',
}} }}
> >
Get demo credentials at kaelio.com/start Get demo credentials at www.kaelio.com/start
<svg <svg
width="14" width="14"
height="14" height="14"
@ -190,6 +191,12 @@ Install the published package globally:
npm install -g @kaelio/ktx npm install -g @kaelio/ktx
``` ```
To upgrade an existing install later, re-run with the `@latest` tag:
```bash
npm install -g @kaelio/ktx@latest
```
**ktx** is open source. If you'd like to hack on it or run from a local checkout, **ktx** is open source. If you'd like to hack on it or run from a local checkout,
the source lives at [github.com/kaelio/ktx](https://github.com/kaelio/ktx) - the source lives at [github.com/kaelio/ktx](https://github.com/kaelio/ktx) -
see [Contributing](/docs/community/contributing) to get set up. see [Contributing](/docs/community/contributing) to get set up.
@ -214,8 +221,8 @@ The wizard walks you through everything **ktx** needs in one pass:
SQLite, PostgreSQL, MySQL, SQL Server, BigQuery, and Snowflake. SQLite, PostgreSQL, MySQL, SQL Server, BigQuery, and Snowflake.
5. **Context sources** - optionally adds dbt, MetricFlow, LookML, Looker, 5. **Context sources** - optionally adds dbt, MetricFlow, LookML, Looker,
Metabase, or Notion. You can skip and add them later. Metabase, or Notion. You can skip and add them later.
6. **Build** - runs the first ingest so semantic sources and wiki pages 6. **Build** - offers to run the first ingest so semantic sources and wiki
are ready for agents. pages are ready for agents. If you skip it, build later with `ktx ingest`.
7. **Agent integration** - installs project-local rules for Claude Code, 7. **Agent integration** - installs project-local rules for Claude Code,
Codex, Cursor, OpenCode, or universal `.agents`. Codex, Cursor, OpenCode, or universal `.agents`.
@ -235,7 +242,7 @@ Testing warehouse
Connection test passed Connection test passed
Building schema context for warehouse Building schema context for warehouse
Running fast database ingest Running database scan
``` ```
If setup exits early, rerun `ktx setup` in the same directory. **ktx** keeps If setup exits early, rerun `ktx setup` in the same directory. **ktx** keeps
@ -246,6 +253,18 @@ progress under `.ktx/setup/` and resumes from the remaining work.
> resuming setup, connecting an agent, checking status, or exploring a > resuming setup, connecting an agent, checking status, or exploring a
> pre-built demo project. > pre-built demo project.
When the wizard finishes, it states where you stand and the single next action:
- **Context built** - **ktx** confirms it is ready for agents and points you to
open your coding agent and ask a data question.
- **Build skipped** - **ktx** tells you setup is complete and that the only step
left is to build context with `ktx ingest`.
Re-running `ktx setup` on an already-configured project goes straight to the
remaining step - building context or connecting an agent - instead of
re-asking every question. Once everything is ready, it confirms you are set
rather than reopening the configuration menu.
## Verify ## Verify
When setup finishes, check readiness: When setup finishes, check readiness:
@ -267,18 +286,41 @@ Agent integration ready: yes (codex:project)
For a structured check inside scripts, use `ktx status --json`. For a structured check inside scripts, use `ktx status --json`.
When setup builds deep context, its final context check looks like: If you skipped the build, `ktx context built` shows `no`. Build it with
`ktx ingest` - there is no need to re-run `ktx setup`.
When setup finishes building context, its final context check looks like:
```text ```text
ktx context is ready for agents. ktx context is ready for agents.
Databases: Databases:
warehouse: deep context complete warehouse: database context complete
Context sources: Context sources:
dbt_main: memory update complete dbt_main: memory update complete
``` ```
Before the build starts, **ktx** runs a live test for every connection the
build depends on. A context build can take several minutes, so if any required
connection is unreachable or misconfigured the build is blocked up front and
**ktx** names the failing connection by id and connector type:
```text
ktx cannot build context: a required connection failed its live test.
Failed connections:
warehouse (postgres)
Each connection must be reachable before ktx builds context.
Run `ktx connection test <id>` to see the error, fix the connection, then retry.
```
Run `ktx connection test <connection-id>` to see the underlying error, fix the
connection, then continue. In interactive setup you can retry without
restarting; with `--no-input` the build exits non-zero and names the failing
connection so scripts can stop early.
## Connect a coding agent ## Connect a coding agent
The setup wizard installs project-local agent rules in the last step. To The setup wizard installs project-local agent rules in the last step. To
@ -296,6 +338,16 @@ separate `ktx` binary on `PATH`. If the CLI path changes, rerun
## What setup writes ## What setup writes
**ktx** writes plain files so people and agents can review changes in git. **ktx** writes plain files so people and agents can review changes in git.
**ktx** initializes a git repository at the project directory and writes context
changes there. If the project directory is nested inside another repository,
**ktx** still keeps its own repo and does not commit to the parent repo.
Because **ktx** owns that repository, it will not adopt one it did not create. If
you point setup at a directory that is already a git repository's root - such as
an existing application checkout - **ktx** stops and asks you to pick a dedicated
directory instead. In the setup wizard choose the **New subfolder** option (for
example `ktx-project`), or pass a fresh `--project-dir` when running setup
non-interactively.
| Path | Purpose | | Path | Purpose |
|------|---------| |------|---------|
@ -325,7 +377,7 @@ ktx setup \
Then build context: Then build context:
```bash ```bash
ktx ingest warehouse --fast ktx ingest warehouse
``` ```
See [ktx setup](/docs/cli-reference/ktx-setup) for the full automation flag See [ktx setup](/docs/cli-reference/ktx-setup) for the full automation flag
@ -338,7 +390,8 @@ surface.
| `ktx: command not found` | Reinstall `@kaelio/ktx` and open a new shell | | `ktx: command not found` | Reinstall `@kaelio/ktx` and open a new shell |
| Setup resumes the wrong project | Pass `--project-dir <path>` | | Setup resumes the wrong project | Pass `--project-dir <path>` |
| LLM or embeddings health check fails | Rerun setup and pick a different credential, model, or backend | | LLM or embeddings health check fails | Rerun setup and pick a different credential, model, or backend |
| Database test fails | Verify the same connection with the database's native client, then rerun setup | | Database test fails | Use the setup recovery menu to retry or re-enter details; if it still fails, verify the same connection with the database's native client |
| Context build blocked: a connection failed its live test | Run `ktx connection test <connection-id>` to see the error, fix the connection, then retry the build |
| Agent integration is incomplete | Run `ktx setup --agents --target <target>` | | Agent integration is incomplete | Run `ktx setup --agents --target <target>` |
## Next steps ## Next steps

View file

@ -24,7 +24,9 @@ external metadata can attach to known warehouse tables.
## Database ingest ## Database ingest
Database ingest records table, column, type, constraint, and row-count context. Database ingest always builds enriched context: tables, columns, types,
constraints, and row counts, plus AI-generated descriptions, embeddings, and
relationship evidence.
```bash ```bash
# Build one configured database connection # Build one configured database connection
@ -34,37 +36,37 @@ ktx ingest warehouse
ktx ingest --all ktx ingest --all
``` ```
Depth controls how much context **ktx** builds: Enriched ingest needs a configured model and embeddings. Run `ktx setup` first;
connections without that configuration fail before any work starts.
| Flag | Best for | What it does | Local-auth backends keep provider credentials out of `ktx.yaml`:
|------|----------|--------------|
| `--fast` | First setup, quick refreshes, CI smoke checks | Deterministic fast ingest with tables, columns, types, constraints, and row counts |
| `--deep` | Agent-ready context for real analysis | Fast ingest plus deep enrichment with descriptions, embeddings, relationship evidence, and optional query history |
Examples:
```bash ```bash
ktx ingest warehouse --fast ktx setup --llm-backend claude-code --no-input
ktx ingest warehouse --deep ktx setup --llm-backend codex --no-input
ktx ingest --all --deep
``` ```
Deep ingest needs LLM and embedding readiness. Otherwise run `ktx setup` or use With `claude-code`, **ktx** agent loops can invoke only the **ktx** MCP tools
`--fast`. for the current run. With `codex`, **ktx** restricts the temporary runtime MCP
server to the current run's tool set, disables Codex web search, requests a
With `claude-code`, **ktx** agent loops can invoke only the **ktx** MCP tools for the read-only sandbox, and sets `approval_policy=never`. The public Codex SDK and
current run. CLI surface may still load user Codex config and built-in command execution or
read-only file capabilities, so use `claude-code` for stricter runtime tool
isolation.
## Query history ## Query history
PostgreSQL, BigQuery, and Snowflake can add query-history context: common joins, PostgreSQL, BigQuery, and Snowflake can add query-history context: common joins,
filters, service-account patterns, redaction rules, and high-usage templates. filters, redaction rules, high-usage templates, and service-account exclusions.
When query history is enabled during setup, **ktx** reviews observed in-scope
roles and can write exact `filters.serviceAccounts` patterns for operational
traffic such as loader or refresh roles.
Enable it during setup, store it under `connections.<id>.context.queryHistory`, Enable it during setup, store it under `connections.<id>.context.queryHistory`,
or request it for one run: or request it for one run:
```bash ```bash
ktx ingest warehouse --deep --query-history ktx ingest warehouse --query-history
# Set the lookback window for BigQuery or Snowflake query history # Set the lookback window for BigQuery or Snowflake query history
ktx ingest warehouse --query-history-window-days 30 ktx ingest warehouse --query-history-window-days 30
``` ```
@ -74,8 +76,8 @@ for one run.
## Relationship evidence ## Relationship evidence
**ktx** scores relationship candidates during supported deep database ingest. The **ktx** scores relationship candidates during database ingest. The public CLI
public CLI does not expose separate relationship review subcommands. does not expose separate relationship review subcommands.
## Context-source ingest ## Context-source ingest
@ -159,7 +161,7 @@ After interactive setup:
```bash ```bash
ktx status ktx status
ktx ingest --all --deep ktx ingest --all
ktx status ktx status
``` ```
@ -176,8 +178,8 @@ ktx wiki "revenue" --json --limit 10
| Symptom | Likely cause | Recovery | | Symptom | Likely cause | Recovery |
|---------|--------------|----------| |---------|--------------|----------|
| Connection not configured | The connection id is missing from `ktx.yaml` | Add it with `ktx setup` | | Connection not configured | The connection id is missing from `ktx.yaml` | Add it with `ktx setup` |
| Deep readiness is missing | LLM or embeddings are not setup-ready | Run `ktx setup`, or rerun with `--fast` | | Enrichment is not configured | LLM or embeddings are not setup-ready | Run `ktx setup` to configure a model and embeddings |
| Query history is unsupported | The selected database driver does not expose query history | Run fast ingest without query-history flags | | Query history is unsupported | The selected database driver does not expose query history | Run ingest without query-history flags |
| No connections configured | The project has no entries under `connections` | Run `ktx setup` and add a database or context-source connection | | No connections configured | The project has no entries under `connections` | Run `ktx setup` and add a database or context-source connection |
| Context-source flags have no effect | Depth and query-history flags were supplied for a context-source connector | Use those flags only for database connections | | Context-source flags have no effect | Query-history flags were supplied for a context-source connector | Use query-history flags only for database connections |
| Text ingest stops early | `--fail-fast` stopped on the first failed item | Fix the item or rerun without `--fail-fast` | | Text ingest stops early | `--fail-fast` stopped on the first failed item | Fix the item or rerun without `--fail-fast` |

View file

@ -16,6 +16,7 @@ Set `llm.provider.backend` to one of these values:
- `gateway`: Use AI Gateway-compatible Anthropic model ids. - `gateway`: Use AI Gateway-compatible Anthropic model ids.
- `claude-code`: Use your local Claude Code session through the Claude Agent - `claude-code`: Use your local Claude Code session through the Claude Agent
SDK. **ktx** strips provider-routing environment variables from child processes. SDK. **ktx** strips provider-routing environment variables from child processes.
- `codex`: Use your local Codex authentication through the Codex SDK.
## Claude Code ## Claude Code
@ -29,24 +30,65 @@ llm:
default: sonnet default: sonnet
triage: haiku triage: haiku
candidateExtraction: sonnet candidateExtraction: sonnet
curator: sonnet curator: opus
reconcile: sonnet reconcile: opus
repair: sonnet repair: haiku
``` ```
During setup, choose the backend interactively or pass the model in automation: During setup, choose the backend interactively or pass it in automation:
```bash ```bash
ktx setup --llm-backend claude-code --llm-model opus --no-input ktx setup --llm-backend claude-code --no-input
``` ```
For Claude Code, `sonnet`, `opus`, and `haiku` map to **ktx** defaults. Full Claude Setup writes `sonnet`, `haiku`, and `opus` aliases into `llm.models`. You can
model IDs are also accepted. edit any role to another alias or a full Claude model ID after setup.
`claude-code` exposes only **ktx** MCP tools for the current agent loop. SDK init `claude-code` exposes only **ktx** MCP tools for the current agent loop. SDK init
metadata may still list host slash commands, skills, and subagents; **ktx** does not metadata may still list host slash commands, skills, and subagents; **ktx** does not
grant execution access to them. grant execution access to them.
## Codex backend
Use `codex` when you want **ktx** to run LLM-backed workflows through your
local Codex authentication instead of a direct provider API key.
```yaml
llm:
provider:
backend: codex
models:
default: gpt-5.5
triage: gpt-5.5
candidateExtraction: gpt-5.5
curator: gpt-5.5
reconcile: gpt-5.5
repair: gpt-5.5
```
Configure it non-interactively:
```bash
ktx setup --llm-backend codex --no-input
```
This is separate from Codex agent-client setup. `ktx setup --agents --target
codex` installs instructions and MCP access for an end-user Codex session.
`ktx setup --llm-backend codex` makes **ktx** itself execute ingest, scan
enrichment, memory, and other LLM-backed work through Codex.
During runtime loops, **ktx** starts a temporary loopback MCP server for the
current run, exposes only the tools passed to that run, asks Codex to use a
read-only sandbox, sets `approval_policy=never`, auto-approves only those
run-scoped MCP tools, and disables Codex web search.
Codex backend isolation is currently limited by the public Codex SDK and CLI
surface. Codex may still load user Codex config and built-in command execution
or read-only file capabilities. Use `llm.provider.backend: claude-code` when
you need stricter Claude-Code-style runtime tool isolation, or remove host
Codex MCP and tool config before running untrusted prompts through the `codex`
backend.
## Prompt caching ## Prompt caching
`llm.promptCaching` has partial parity on `claude-code`. Status and doctor warn `llm.promptCaching` has partial parity on `claude-code`. Status and doctor warn

View file

@ -61,11 +61,14 @@ committing the file.
## A typical review session ## A typical review session
The loop above describes the shape. In practice, one review session looks like The loop above describes the shape. Run these commands from the **ktx** project
this: directory. **ktx** keeps that directory as its own git repository, even when the
directory lives inside another repository, so reviewing context changes never
requires committing to a parent application repo.
```bash ```bash
# 1. Run ingest on a branch # 1. Run ingest on a branch
cd /path/to/ktx-project
git checkout -b ingest/2026-05-21 git checkout -b ingest/2026-05-21
ktx ingest --all ktx ingest --all

View file

@ -111,12 +111,13 @@ non-obvious terms.
Agents can refresh context when the user asks them to: Agents can refresh context when the user asks them to:
```bash ```bash
ktx ingest warehouse --fast ktx ingest warehouse
ktx ingest ktx ingest
ktx ingest --file docs/revenue-notes.md --connection-id warehouse ktx ingest --file docs/revenue-notes.md --connection-id warehouse
``` ```
Use `--deep` only when LLM and embedding setup is ready. Database ingest builds enriched context and requires a configured model and
embeddings; run `ktx setup` first if they are not ready.
## Good agent behavior ## Good agent behavior

View file

@ -44,12 +44,17 @@ Use this order for most context changes:
Semantic sources are YAML files for queryable tables or custom SQL. They define Semantic sources are YAML files for queryable tables or custom SQL. They define
agent-facing measures, dimensions, segments, joins, and grain. agent-facing measures, dimensions, segments, joins, and grain.
Semantic source files live at: Semantic source files live under:
```text ```text
semantic-layer/<connection-id>/<source-name>.yaml semantic-layer/<connection-id>/
``` ```
The file's `name:` field is the source's identity — it carries the warehouse
identifier verbatim, including case. The filename is a derived label: simple
lowercase names get `<source-name>.yaml`, anything else gets a slugged
filename. Renaming a file does not rename the source.
### Minimal source ### Minimal source
```yaml ```yaml
@ -152,7 +157,7 @@ joins:
| Field | Required | Description | | Field | Required | Description |
|-------|----------|-------------| |-------|----------|-------------|
| `name` | Yes | Source identifier. Use lowercase words and underscores. | | `name` | Yes | Source identity (not the filename). When overlaying an ingested table, match the manifest identifier verbatim, including case (e.g. `SIGNED_UP`); for a new standalone source, lowercase words and underscores are recommended. |
| `descriptions` | No | Description map keyed by source, such as `user`, `dbt`, or `ai`. | | `descriptions` | No | Description map keyed by source, such as `user`, `dbt`, or `ai`. |
| `table` or `sql` | Yes | Database table or custom SQL expression. Use exactly one. | | `table` or `sql` | Yes | Database table or custom SQL expression. Use exactly one. |
| `grain` | Yes | Columns that uniquely identify a row at the source grain. | | `grain` | Yes | Columns that uniquely identify a row at the source grain. |

View file

@ -68,19 +68,30 @@ If you choose an install mode, it then asks which targets to install:
``` ```
When every selected target supports both project and global setup, the command When at least one selected target supports project-scoped setup, the command
also asks where to install supported agent config: asks where to install agent config:
```txt ```txt
◆ Where should ktx install supported agent config? ◆ Where should ktx install agent config?
│ ktx project: /path/to/your/ktx-project │ ktx project: /path/to/your/ktx-project
│ ○ Project scope (ktx project directory) │ ○ ktx project directory /path/to/your/ktx-project
│ ○ Current directory /path/to/where/you/ran/ktx
│ ○ Custom directory… (enter a path)
│ ○ Global scope (user config) │ ○ Global scope (user config)
``` ```
The first three choices write project-scoped files (`.claude/`, `.mcp.json`,
`.cursor/`, skills, and rules) into the chosen directory while still pointing
them at this ktx project. Use **Current directory** or **Custom directory…**
when you open your coding agent from somewhere other than the ktx project
directory. **Current directory** is hidden when it is already the ktx project
directory, and **Global scope** appears only when every selected target
supports global setup. Non-interactive runs pass `--install-dir <path>` (for
example `--install-dir .`) for the same result.
## Generated files ## Generated files
**ktx** writes MCP client configuration and analytics guidance by default. It writes **ktx** writes MCP client configuration and analytics guidance by default. It writes

View file

@ -38,15 +38,16 @@ LookML uses top-level `repoUrl`, and MetricFlow uses nested
## dbt ## dbt
Ingests schema definitions, model descriptions, column metadata, and test coverage from a dbt project. Ingests schema definitions, model descriptions, column metadata, and column test definitions from a dbt project.
### What it provides ### What it provides
- Model and source definitions from `schema.yml` files - Model and source definitions from `schema.yml` files
- Column descriptions and types - Column names, descriptions, and data types
- Test coverage signals - Column tests, mapped to semantic facts — `not_null` / `unique` become column constraints, `accepted_values` becomes enum value lists, and `relationships` becomes join / foreign-key edges
- Semantic model references (if using dbt semantic layer) - Model and source tags, and source freshness settings
- Data lineage between models
MetricFlow `semantic_models:` and `metrics:` are ingested through the separate [MetricFlow](#metricflow) source, not the dbt driver.
### Connection config ### Connection config
@ -87,9 +88,9 @@ connections:
### What gets ingested ### What gets ingested
- YAML semantic sources generated from dbt schema files - **Semantic-layer overlays** (`semantic-layer/*.yaml`): descriptions, constraints, enum values, and joins from the dbt YAML are written onto the semantic source for the matching warehouse table. Overlays land on the warehouse connection that owns the table, which is usually a different connection than the dbt source itself.
- One work unit per semantic source (for projects with >25 YAML files) or all at once for smaller projects - **Wiki pages** (`wiki/`): for definitions or relationships that don't map to a confirmed physical table.
- Column descriptions, tests, and relationships are preserved - **Work units** for parallel processing: one per schema file under `models/` when the project has more than 25 YAML files, otherwise a single combined unit.
--- ---
@ -101,7 +102,7 @@ Ingests MetricFlow semantic models and metric definitions. Useful when your team
- Semantic model definitions (entities, dimensions, measures) - Semantic model definitions (entities, dimensions, measures)
- Cross-model metric definitions - Cross-model metric definitions
- Dimension and entity relationships between models - Entity relationships between models, inferred from matching foreign and primary entities
### Connection config ### Connection config
@ -133,7 +134,7 @@ For a local path:
### What gets ingested ### What gets ingested
- Semantic models with their entities, dimensions, and measures - Semantic models with their entities, dimensions, measures, and the join edges inferred from entity relationships
- Metric definitions with their expressions and filters - Metric definitions with their expressions and filters
- Work units organized by connected component (metrics + related semantic models grouped together) - Work units organized by connected component (metrics + related semantic models grouped together)
@ -178,10 +179,10 @@ For a local path:
### What gets ingested ### What gets ingested
- View and model definitions organized by connected component - One work unit per model, plus a unit for orphan views and one per dashboard
- LookML field types mapped to semantic layer column types - Semantic-layer sources per view — overlays for thin `sql_table_name` wrappers, standalone sources for `derived_table` views
- Join definitions and relationship cardinalities - Measures, joins (with their Looker `relationship:`), and field types mapped to column types (`yesno` → boolean, date/timestamp → time)
- SQL table references for warehouse mapping validation - Wiki pages for relationships and descriptions, with warehouse identifiers verified before writing
### Warehouse mapping ### Warehouse mapping
@ -192,19 +193,19 @@ Optionally validate that LookML references match your expected Looker connection
expectedLookerConnectionName: postgres_connection expectedLookerConnectionName: postgres_connection
``` ```
This validates that LookML model `connection:` declarations match expectations, flagging mismatches during ingestion. This compares each model's `connection:` declaration against the expected name. Mismatched models are flagged, and semantic-layer writes are disabled for them during that ingest while wiki extraction still proceeds.
--- ---
## Metabase ## Metabase
Ingests dashboards, questions, and their underlying SQL queries from a Metabase instance. Maps Metabase databases to your **ktx** warehouse connections. Ingests collections, questions, models, and metrics — with their underlying SQL — from a Metabase instance. Maps Metabase databases to your **ktx** warehouse connections.
### What it provides ### What it provides
- Dashboard metadata and organization - Collections and their hierarchy, used to organize ingested context
- Question/query definitions (native SQL and structured queries) - Questions, models, and metrics — resolved SQL for both native and structured (MBQL) queries
- Table and column usage patterns from queries - Each card's output schema: column types and primary/foreign-key hints
- Database-to-warehouse relationship mapping - Database-to-warehouse relationship mapping
### Connection config ### Connection config
@ -233,9 +234,9 @@ Generate an API key in Metabase: **Admin > Settings > Authentication > API Keys*
### What gets ingested ### What gets ingested
- Semantic sources generated from SQL queries in questions - Semantic-layer sources generated from each card's resolved SQL and column metadata, written to the mapped warehouse connection
- Wiki pages for dashboards (purpose, key metrics, relationships) - Fallback wiki notes only when a referenced table can't be mapped or an identifier can't be verified
- Work units per dashboard and per question - One work unit per Metabase collection; re-syncs reprocess only collections with changed cards
### Warehouse mapping ### Warehouse mapping
@ -289,10 +290,10 @@ Generate API credentials in Looker: **Admin > Users > Edit > API Keys**.
### What gets ingested ### What gets ingested
- Semantic sources from explore field definitions - Semantic-layer sources from explore fields, written to the mapped warehouse connection (mapped explores only)
- Wiki pages for dashboards (purpose, audience, key metrics) - Wiki pages capturing reusable metric, segment, and domain knowledge from dashboards and Looks
- Triage signals for automated content classification - Usage and recency signals that drive a triage gate, focusing processing on high-value content
- Work units per explore and per dashboard - Work units per explore, per dashboard, and per Look
### Warehouse mapping ### Warehouse mapping
@ -314,10 +315,10 @@ Ingests pages and databases from a Notion workspace as wiki pages. Useful for ca
### What it provides ### What it provides
- Wiki pages synthesized from Notion content - Notion pages crawled from selected roots or all accessible content
- Page hierarchy and relationships - Page bodies and blocks normalized to Markdown
- Database schemas (when Notion databases describe primary sources) - Page hierarchy and cross-page links (child pages, mentions, relations)
- Semantic clustering for organized ingestion - Notion databases and their data-source rows as individual pages
### Connection config ### Connection config
@ -356,6 +357,7 @@ Create an integration at [notion.so/my-integrations](https://www.notion.so/my-in
| `crawl_mode` | `all_accessible` or `selected_roots` | - | | `crawl_mode` | `all_accessible` or `selected_roots` | - |
| `root_page_ids` | Page IDs to crawl from (for `selected_roots`) | `[]` | | `root_page_ids` | Page IDs to crawl from (for `selected_roots`) | `[]` |
| `root_database_ids` | Database IDs to include | `[]` | | `root_database_ids` | Database IDs to include | `[]` |
| `root_data_source_ids` | Data-source IDs to include (for `selected_roots`) | `[]` |
| `max_pages_per_run` | Pages processed per sync | `1000` | | `max_pages_per_run` | Pages processed per sync | `1000` |
| `max_knowledge_creates_per_run` | New pages created per sync | `25` | | `max_knowledge_creates_per_run` | New pages created per sync | `25` |
| `max_knowledge_updates_per_run` | Pages updated per sync | `20` | | `max_knowledge_updates_per_run` | Pages updated per sync | `20` |
@ -363,13 +365,13 @@ Create an integration at [notion.so/my-integrations](https://www.notion.so/my-in
### What gets ingested ### What gets ingested
- Wiki pages synthesized from Notion content (not raw copies) - Wiki pages synthesized from Notion content (not raw copies)
- Domain context extracted and organized by topic - Semantic-layer sources when a page defines a reusable dataset or metric mapped to a confirmed non-Notion target; otherwise the fact stays wiki-only
- Triage signals for classifying page relevance - Page-relevance triage that skips transient content (task lists, status updates, date-titled snapshots)
- Work units clustered by semantic similarity for efficient processing - Work units clustered by embedding similarity for efficient synthesis
### Notes ### Notes
- Notion is knowledge-only - it does not produce semantic layer sources - Notion is wiki-first: it writes durable wiki pages by default and only emits semantic-layer sources for content mapped to a confirmed non-Notion target; unmapped facts stay wiki-only
- Rate limits apply; large workspaces may require multiple ingestion runs - Rate limits apply; large workspaces may require multiple ingestion runs
- Incremental sync cursors are stored in `.ktx/db.sqlite`; don't add - Incremental sync cursors are stored in `.ktx/db.sqlite`; don't add
`last_successful_cursor` to `ktx.yaml` `last_successful_cursor` to `ktx.yaml`

View file

@ -517,5 +517,5 @@ No authentication required - SQLite is file-based. The file must be readable by
| Connection URL appears in git diff | A literal credential URL was written to `ktx.yaml` | Replace it with `env:NAME` or `file:/path/to/secret` and rotate exposed credentials | | Connection URL appears in git diff | A literal credential URL was written to `ktx.yaml` | Replace it with `env:NAME` or `file:/path/to/secret` and rotate exposed credentials |
| Database ingest returns no tables | Schema, database, or project filter is wrong, or the user lacks metadata permissions | Verify the schema list and grant metadata read permissions | | Database ingest returns no tables | Schema, database, or project filter is wrong, or the user lacks metadata permissions | Verify the schema list and grant metadata read permissions |
| Query history is empty | Query history extension or warehouse history view is unavailable | Enable the warehouse-specific history feature, then rerun `ktx ingest <connectionId> --query-history` or `ktx setup` | | Query history is empty | Query history extension or warehouse history view is unavailable | Enable the warehouse-specific history feature, then rerun `ktx ingest <connectionId> --query-history` or `ktx setup` |
| Column statistics are missing | Connector cannot access stats tables or the warehouse does not expose them | Grant stats permissions where supported; otherwise rely on fast schema context | | Column statistics are missing | Connector cannot access stats tables or the warehouse does not expose them | Grant stats permissions where supported; otherwise rely on schema-level context without column statistics |
| Semantic query execution fails | Connection is missing, unreachable, or query execution is disabled | Run `ktx connection test <id>` and check the `ktx sl query` flags | | Semantic query execution fails | Connection is missing, unreachable, or query execution is disabled | Run `ktx connection test <id>` and check the `ktx sl query` flags |

View file

@ -8,7 +8,6 @@
"integrations", "integrations",
"configuration", "configuration",
"cli-reference", "cli-reference",
"ai-resources",
"community" "community"
] ]
} }

View file

@ -54,9 +54,7 @@ ktx provides semantic-layer files, warehouse scans, wiki pages, provenance, and
- Installable setup skill: run \`npx skills add Kaelio/ktx --skill ktx\` from - Installable setup skill: run \`npx skills add Kaelio/ktx --skill ktx\` from
the project you want to configure. the project you want to configure.
${link("/docs/ai-resources/agent-quickstart", "Agent Quickstart", "Task-first route for coding assistants using ktx")} ${link("/docs/community/ai-resources", "AI Resources", "How coding agents read, cite, and act on the ktx docs")}
${link("/docs/ai-resources/markdown-access", "Markdown Access", "Fetch ktx docs as llms.txt, llms-full.txt, or per-page Markdown")}
${link("/docs/ai-resources/agent-instructions", "Agent Instructions", "Suggested instructions for coding assistants that need to read and cite ktx docs")}
## Start Here ## Start Here
@ -67,7 +65,7 @@ ${link("/docs/guides/writing-context", "Writing Context", "Write semantic source
## Machine-Readable Documentation ## Machine-Readable Documentation
- [Full documentation](${absoluteUrl("/llms-full.txt")}): All docs pages in one plain-text markdown response - [Full documentation](${absoluteUrl("/llms-full.txt")}): All docs pages in one plain-text markdown response
- [Markdown access guide](${absoluteUrl("/docs/ai-resources/markdown-access.md")}): How to fetch llms.txt, llms-full.txt, and per-page Markdown - [AI Resources guide](${absoluteUrl("/docs/community/ai-resources.md")}): How agents fetch llms.txt, llms-full.txt, and per-page Markdown
- [Quickstart markdown](${absoluteUrl("/docs/getting-started/quickstart.md")}): Human setup walkthrough - [Quickstart markdown](${absoluteUrl("/docs/getting-started/quickstart.md")}): Human setup walkthrough
- [Semantic-layer CLI markdown](${absoluteUrl("/docs/cli-reference/ktx-sl.md")}): Semantic-layer commands and JSON output - [Semantic-layer CLI markdown](${absoluteUrl("/docs/cli-reference/ktx-sl.md")}): Semantic-layer commands and JSON output
- [Wiki CLI markdown](${absoluteUrl("/docs/cli-reference/ktx-wiki.md")}): Wiki page commands and JSON output - [Wiki CLI markdown](${absoluteUrl("/docs/cli-reference/ktx-wiki.md")}): Wiki page commands and JSON output
@ -147,8 +145,8 @@ function absoluteUrl(path: string) {
function formatCategoryName(category: string) { function formatCategoryName(category: string) {
const labels: Record<string, string> = { const labels: Record<string, string> = {
"ai-resources": "AI Resources",
"cli-reference": "CLI Reference", "cli-reference": "CLI Reference",
community: "Community & Resources",
}; };
if (labels[category]) { if (labels[category]) {

View file

@ -30,7 +30,36 @@ const config = {
}; };
}, },
async redirects() { async redirects() {
// Alias-host canonicalization MUST come before the generic root/docs
// redirects below. Those generic rules have no host guard, so if they ran
// first they would inject a "/ktx" basePath into the path on the alias
// hosts, which the alias catch-alls would then prepend a second time —
// producing https://docs.kaelio.com/ktx/ktx/docs/... Redirects also run
// before beforeFiles rewrites, so the ktx.sh catch-all must exclude
// /stars* to let the stars dashboard rewrite proxy through.
return [ return [
{
source: "/slack",
has: [{ type: "host", value: "ktx.sh" }],
destination:
"https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ",
permanent: false,
basePath: false,
},
{
source: "/:path*",
has: [{ type: "host", value: "docs.ktx.sh" }],
destination: "https://docs.kaelio.com/ktx/:path*",
permanent: true,
basePath: false,
},
{
source: "/:path((?!stars(?:/|$)).*)",
has: [{ type: "host", value: "ktx.sh" }],
destination: "https://docs.kaelio.com/ktx/:path",
permanent: true,
basePath: false,
},
{ {
source: "/", source: "/",
destination: "/ktx/docs/getting-started/introduction", destination: "/ktx/docs/getting-started/introduction",
@ -44,26 +73,30 @@ const config = {
basePath: false, basePath: false,
}, },
{ {
source: "/:path*", // AI Resources collapsed from four pages to one and now lives under the
has: [{ type: "host", value: "docs.ktx.sh" }], // Community & Resources section. Redirect the old top-level URL and the
destination: "https://docs.kaelio.com/ktx/:path*", // retired per-page slugs to the new home. Redirects run before the .md
// rewrite, so the Markdown variants must be matched first and keep their
// .md suffix; otherwise a cached Markdown URL would 308 to the HTML page
// and break the agent Markdown contract.
source: "/docs/ai-resources.md",
destination: "/docs/community/ai-resources.md",
permanent: true, permanent: true,
basePath: false,
}, },
{ {
source: "/slack", source: "/docs/ai-resources/:slug([^/]+\\.md)",
has: [{ type: "host", value: "ktx.sh" }], destination: "/docs/community/ai-resources.md",
destination: permanent: true,
"https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ",
permanent: false,
basePath: false,
}, },
{ {
source: "/:path((?!stars(?:/|$)).*)", source: "/docs/ai-resources",
has: [{ type: "host", value: "ktx.sh" }], destination: "/docs/community/ai-resources",
destination: "https://docs.kaelio.com/ktx/:path", permanent: true,
},
{
source: "/docs/ai-resources/:slug",
destination: "/docs/community/ai-resources",
permanent: true, permanent: true,
basePath: false,
}, },
]; ];
}, },

View file

@ -12,15 +12,16 @@
"dependencies": { "dependencies": {
"@xyflow/react": "^12.10.2", "@xyflow/react": "^12.10.2",
"fumadocs-core": "16.8.10", "fumadocs-core": "16.8.10",
"fumadocs-mdx": "15.0.4", "fumadocs-mdx": "15.0.7",
"fumadocs-ui": "16.8.10", "fumadocs-ui": "16.8.10",
"html-to-image": "1.11.11",
"next": "^16", "next": "^16",
"react": "19.2.6", "react": "19.2.6",
"react-dom": "19.2.6" "react-dom": "19.2.6"
}, },
"devDependencies": { "devDependencies": {
"@tailwindcss/postcss": "^4", "@tailwindcss/postcss": "^4",
"@types/node": "^25.7.0", "@types/node": "^25.9.1",
"@types/react": "^19", "@types/react": "^19",
"@types/react-dom": "^19", "@types/react-dom": "^19",
"tailwindcss": "^4", "tailwindcss": "^4",

View file

@ -1,210 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1346" height="1710" viewBox="0 0 1346 1710" role="img" aria-labelledby="title desc">
<title id="title">ktx ingestion flow</title>
<desc id="desc">Source systems flow through source connectors, context builder, reconciliation, and validation to create wiki Markdown and semantic-layer YAML outputs.</desc>
<defs>
<filter id="card-shadow" x="-12%" y="-12%" width="124%" height="124%" color-interpolation-filters="sRGB">
<feDropShadow dx="0" dy="2" stdDeviation="2" flood-color="#0f172a" flood-opacity="0.14"/>
</filter>
<filter id="dark-shadow" x="-12%" y="-12%" width="124%" height="124%" color-interpolation-filters="sRGB">
<feDropShadow dx="0" dy="2" stdDeviation="2" flood-color="#020617" flood-opacity="0.22"/>
</filter>
<filter id="glow-blue" x="-160%" y="-160%" width="420%" height="420%">
<feGaussianBlur stdDeviation="7" result="blur"/>
<feMerge>
<feMergeNode in="blur"/>
<feMergeNode in="SourceGraphic"/>
</feMerge>
</filter>
<marker id="arrow" viewBox="0 0 10 10" refX="8.5" refY="5" markerWidth="9" markerHeight="9" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#94a3b8"/>
</marker>
<style>
.card { fill: #ffffff; stroke: #e2e8f0; stroke-width: 1.4; filter: url(#card-shadow); }
.stage { fill: #0b1f23; stroke: #17343a; stroke-width: 1.2; filter: url(#dark-shadow); }
.title { fill: #24272d; font: 700 28px Inter, Arial, sans-serif; }
.body { fill: #666b73; font: 500 18px Inter, Arial, sans-serif; }
.tag { fill: #6b7280; font: 500 16px Inter, Arial, sans-serif; }
.mono { font: 700 20px "SFMono-Regular", Consolas, monospace; }
.stage-title { fill: #f8fafc; font: 700 28px Inter, Arial, sans-serif; }
.stage-body { fill: #b8c6ca; font: 500 20px Inter, Arial, sans-serif; }
.index { fill: #07313a; font: 700 22px Inter, Arial, sans-serif; text-anchor: middle; dominant-baseline: middle; }
.edge { fill: none; stroke: #94a3b8; stroke-width: 2; stroke-linecap: round; stroke-linejoin: round; }
.dash { fill: none; stroke: #64748b; stroke-width: 1.8; stroke-dasharray: 5 8; stroke-linecap: round; }
</style>
</defs>
<g id="source-cards">
<g transform="translate(24 39)">
<rect class="card" x="0" y="0" width="298" height="285" rx="4"/>
<rect x="0" y="0" width="298" height="4" rx="2" fill="#3b82f6"/>
<text class="title" x="22" y="52">Databases</text>
<text class="body" x="22" y="92">Schemas, columns, keys,</text>
<text class="body" x="22" y="120">row counts, and query</text>
<text class="body" x="22" y="148">history.</text>
<g transform="translate(22 180)">
<rect x="0" y="0" width="112" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">PostgreSQL</text>
<rect x="120" y="0" width="100" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="132" y="24">Snowflake</text>
<rect x="0" y="46" width="92" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="70">BigQuery</text>
<rect x="100" y="46" width="74" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="112" y="70">SQLite</text>
</g>
</g>
<g transform="translate(358 39)">
<rect class="card" x="0" y="0" width="298" height="285" rx="4"/>
<rect x="0" y="0" width="298" height="4" rx="2" fill="#f97316"/>
<text class="title" x="22" y="52">BI tools</text>
<text class="body" x="22" y="92">Dashboards, questions,</text>
<text class="body" x="22" y="120">explores, usage, and trusted</text>
<text class="body" x="22" y="148">examples.</text>
<g transform="translate(22 180)">
<rect x="0" y="0" width="96" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">Metabase</text>
<rect x="104" y="0" width="74" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="116" y="24">Looker</text>
</g>
</g>
<g transform="translate(692 39)">
<rect class="card" x="0" y="0" width="298" height="285" rx="4"/>
<rect x="0" y="0" width="298" height="4" rx="2" fill="#f59e0b"/>
<text class="title" x="22" y="52">Modeling code</text>
<text class="body" x="22" y="92">Existing metrics, dimensions,</text>
<text class="body" x="22" y="120">models, joins, and entities.</text>
<g transform="translate(22 152)">
<rect x="0" y="0" width="48" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">dbt</text>
<rect x="56" y="0" width="82" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="68" y="24">LookML</text>
<rect x="0" y="46" width="102" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="70">MetricFlow</text>
</g>
</g>
<g transform="translate(1026 39)">
<rect class="card" x="0" y="0" width="298" height="285" rx="4"/>
<rect x="0" y="0" width="298" height="4" rx="2" fill="#10b981"/>
<text class="title" x="22" y="52">Docs and notes</text>
<text class="body" x="22" y="92">Policies, caveats, team</text>
<text class="body" x="22" y="120">definitions, and analyst</text>
<text class="body" x="22" y="148">context.</text>
<g transform="translate(22 180)">
<rect x="0" y="0" width="72" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">Notion</text>
<rect x="80" y="0" width="84" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="92" y="24">Any text</text>
</g>
</g>
</g>
<g id="edges">
<path class="edge" d="M172 324 V380 Q172 394 186 394 H507 Q507 394 507 380 V324"/>
<path class="edge" d="M841 324 V380 Q841 394 827 394 H507"/>
<path class="edge" d="M1175 324 V380 Q1175 394 1161 394 H673 Q673 394 673 408 V433" marker-end="url(#arrow)"/>
<path class="edge" d="M507 394 H673"/>
<path class="edge" d="M673 618 V651" marker-end="url(#arrow)"/>
<path class="edge" d="M673 833 V866" marker-end="url(#arrow)"/>
<path class="edge" d="M673 1048 V1081" marker-end="url(#arrow)"/>
<path class="edge" d="M673 1262 V1310 Q673 1325 656 1325 H305 Q291 1325 291 1339 V1364" marker-end="url(#arrow)"/>
<path class="edge" d="M673 1262 V1310 Q673 1325 690 1325 H1043 Q1057 1325 1057 1339 V1364" marker-end="url(#arrow)"/>
<path class="dash" d="M546 1523 H800"/>
<path d="M546 1523 l9 -6 v12 z" fill="#64748b"/>
<path d="M800 1523 l-9 -6 v12 z" fill="#64748b"/>
</g>
<g id="particles">
<circle cx="256" cy="394" r="18" fill="#3b82f6" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="256" cy="394" r="6" fill="#3b82f6" opacity="0.9"/>
<circle cx="632" cy="394" r="18" fill="#f97316" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="632" cy="394" r="6" fill="#f97316" opacity="0.9"/>
<circle cx="830" cy="394" r="18" fill="#10b981" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="830" cy="394" r="6" fill="#10b981" opacity="0.9"/>
<circle cx="673" cy="635" r="17" fill="#10b981" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="673" cy="635" r="6" fill="#10b981" opacity="0.9"/>
<circle cx="673" cy="1065" r="17" fill="#f59e0b" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="673" cy="1065" r="6" fill="#f59e0b" opacity="0.9"/>
<circle cx="573" cy="1322" r="17" fill="#3b82f6" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="573" cy="1322" r="6" fill="#3b82f6" opacity="0.9"/>
</g>
<g id="stages">
<g transform="translate(464 438)">
<rect class="stage" x="0" y="0" width="420" height="180" rx="4"/>
<circle cx="52" cy="90" r="26" fill="#55dced"/>
<text class="index" x="52" y="90">1</text>
<text class="stage-title" x="98" y="72">Source connectors</text>
<text class="stage-body" x="98" y="110">Read each configured system in</text>
<text class="stage-body" x="98" y="140">its native shape.</text>
</g>
<g transform="translate(464 653)">
<rect class="stage" x="0" y="0" width="420" height="180" rx="4"/>
<circle cx="52" cy="90" r="26" fill="#55dced"/>
<text class="index" x="52" y="90">2</text>
<text class="stage-title" x="98" y="72">Context builder</text>
<text class="stage-body" x="98" y="110">Turn source evidence into</text>
<text class="stage-body" x="98" y="140">proposed context updates.</text>
</g>
<g transform="translate(464 868)">
<rect class="stage" x="0" y="0" width="420" height="180" rx="4"/>
<circle cx="52" cy="90" r="26" fill="#55dced"/>
<text class="index" x="52" y="90">3</text>
<text class="stage-title" x="98" y="72">Reconciliation</text>
<text class="stage-body" x="98" y="110">Merge new evidence with the</text>
<text class="stage-body" x="98" y="140">context that already exists.</text>
</g>
<g transform="translate(464 1082)">
<rect class="stage" x="0" y="0" width="420" height="180" rx="4"/>
<circle cx="52" cy="90" r="26" fill="#55dced"/>
<text class="index" x="52" y="90">4</text>
<text class="stage-title" x="98" y="72">Validation</text>
<text class="stage-body" x="98" y="110">Check references and semantics</text>
<text class="stage-body" x="98" y="140">before agents rely on them.</text>
</g>
</g>
<g id="outputs">
<g transform="translate(60 1373)">
<rect class="card" x="0" y="0" width="485" height="329" rx="4"/>
<rect x="0" y="0" width="485" height="4" rx="2" fill="#10b981"/>
<text class="mono" x="24" y="52" fill="#10b981">wiki/*.md</text>
<text class="title" x="24" y="100">Wiki</text>
<g transform="translate(24 122)">
<rect x="0" y="0" width="90" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">free-form</text>
<rect x="98" y="0" width="140" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="110" y="24">auto-maintained</text>
</g>
<text class="body" x="24" y="194">Definitions, caveats, policies, analyst notes, and</text>
<text class="body" x="24" y="222">business language that agents can search.</text>
</g>
<g transform="translate(803 1373)">
<rect class="card" x="0" y="0" width="485" height="329" rx="4"/>
<rect x="0" y="0" width="485" height="4" rx="2" fill="#3b82f6"/>
<text class="mono" x="24" y="52" fill="#3b82f6">semantic-layer/*.yaml</text>
<text class="title" x="24" y="100">Semantic layer</text>
<g transform="translate(24 122)">
<rect x="0" y="0" width="96" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">structured</text>
<rect x="104" y="0" width="104" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="116" y="24">executable</text>
<rect x="216" y="0" width="140" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="228" y="24">auto-maintained</text>
</g>
<text class="body" x="24" y="194">Metrics, joins, tables, dimensions, filters, and</text>
<text class="body" x="24" y="222">segments that ktx can validate and compile into</text>
<text class="body" x="24" y="250">SQL.</text>
</g>
<g transform="translate(622 1505)">
<rect x="0" y="0" width="102" height="36" rx="4" fill="#ffffff" stroke="#e5e1dc"/>
<text class="tag" x="13" y="24">references</text>
</g>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 137 KiB

After

Width:  |  Height:  |  Size: 346 KiB

Before After
Before After

Binary file not shown.

After

Width:  |  Height:  |  Size: 176 KiB

View file

@ -2,6 +2,8 @@ import assert from "node:assert/strict";
import { spawn } from "node:child_process"; import { spawn } from "node:child_process";
import { once } from "node:events"; import { once } from "node:events";
import { readFile, writeFile } from "node:fs/promises"; import { readFile, writeFile } from "node:fs/promises";
import http from "node:http";
import https from "node:https";
import { dirname, join } from "node:path"; import { dirname, join } from "node:path";
import { createServer } from "node:net"; import { createServer } from "node:net";
import { after, before, test } from "node:test"; import { after, before, test } from "node:test";
@ -100,6 +102,37 @@ after(async () => {
} }
}); });
// Node's fetch (undici) overwrites the Host header with the connection host,
// so the alias-host redirect rules never match. The low-level http(s) client
// sends Host verbatim, which is what the alias canonicalization keys off of.
function requestWithHost(hostHeader, path) {
const target = new URL(docsSiteUrl);
const client = target.protocol === "https:" ? https : http;
const port =
target.port || (target.protocol === "https:" ? "443" : "80");
return new Promise((resolve, reject) => {
const request = client.request(
{
hostname: target.hostname,
port,
path,
method: "GET",
headers: { Host: hostHeader },
},
(response) => {
response.resume();
resolve({
status: response.statusCode,
location: response.headers.location,
});
},
);
request.on("error", reject);
request.end();
});
}
test("/ktx/docs redirects to the docs introduction", async () => { test("/ktx/docs redirects to the docs introduction", async () => {
const response = await fetch(`${docsSiteUrl}${docsBasePath}/docs`, { const response = await fetch(`${docsSiteUrl}${docsBasePath}/docs`, {
redirect: "manual", redirect: "manual",
@ -112,6 +145,53 @@ test("/ktx/docs redirects to the docs introduction", async () => {
); );
}); });
test("retired AI Resources URLs redirect to the page under Community", async () => {
// The former top-level URL.
const bare = await fetch(
`${docsSiteUrl}${docsBasePath}/docs/ai-resources`,
{ redirect: "manual" },
);
assert.equal(bare.status, 308);
assert.equal(
bare.headers.get("location"),
`${docsBasePath}/docs/community/ai-resources`,
);
// A retired per-page slug.
const slug = await fetch(
`${docsSiteUrl}${docsBasePath}/docs/ai-resources/agent-quickstart`,
{ redirect: "manual" },
);
assert.equal(slug.status, 308);
assert.equal(
slug.headers.get("location"),
`${docsBasePath}/docs/community/ai-resources`,
);
// A retired per-page Markdown URL must stay Markdown: it has to redirect to
// the new .md route, not fall through to the HTML page.
const markdown = await fetch(
`${docsSiteUrl}${docsBasePath}/docs/ai-resources/agent-quickstart.md`,
{ redirect: "manual" },
);
assert.equal(markdown.status, 308);
assert.equal(
markdown.headers.get("location"),
`${docsBasePath}/docs/community/ai-resources.md`,
);
// Following that redirect end to end must land on Markdown, not HTML.
const followed = await fetch(
`${docsSiteUrl}${docsBasePath}/docs/ai-resources/agent-quickstart.md`,
);
assert.equal(followed.status, 200);
assert.match(followed.headers.get("content-type") ?? "", /text\/markdown/);
});
test("/ redirects into the /ktx docs site", async () => { test("/ redirects into the /ktx docs site", async () => {
const response = await fetch(`${docsSiteUrl}/`, { const response = await fetch(`${docsSiteUrl}/`, {
redirect: "manual", redirect: "manual",
@ -141,3 +221,51 @@ test("/ktx/api/search returns docs search results", async () => {
"search should return at least one docs result", "search should return at least one docs result",
); );
}); });
test("ktx.sh canonicalizes to a single /ktx basePath on the docs host", async () => {
const root = await requestWithHost("ktx.sh", "/");
assert.equal(root.status, 308);
assert.equal(root.location, "https://docs.kaelio.com/ktx/");
assert.ok(
!root.location.includes("/ktx/ktx"),
"the basePath must not be doubled",
);
const page = await requestWithHost(
"ktx.sh",
"/docs/getting-started/quickstart",
);
assert.equal(page.status, 308);
assert.equal(
page.location,
"https://docs.kaelio.com/ktx/docs/getting-started/quickstart",
);
});
test("docs.ktx.sh canonicalizes to a single /ktx basePath on the docs host", async () => {
const root = await requestWithHost("docs.ktx.sh", "/");
assert.equal(root.status, 308);
assert.equal(root.location, "https://docs.kaelio.com/ktx");
assert.ok(
!root.location.includes("/ktx/ktx"),
"the basePath must not be doubled",
);
const page = await requestWithHost("docs.ktx.sh", "/llms.txt");
assert.equal(page.status, 308);
assert.equal(page.location, "https://docs.kaelio.com/ktx/llms.txt");
});
test("ktx.sh keeps the /slack and /stars exceptions", async () => {
const slack = await requestWithHost("ktx.sh", "/slack");
assert.equal(slack.status, 307);
assert.match(slack.location, /^https:\/\/join\.slack\.com\//);
// /stars is proxied by a beforeFiles rewrite, so the apex catch-all must not
// canonicalize it to the docs host.
const stars = await requestWithHost("ktx.sh", "/stars");
assert.ok(
!(stars.location ?? "").startsWith("https://docs.kaelio.com"),
"the stars dashboard must not be redirected to the docs host",
);
});

View file

@ -85,7 +85,7 @@ test("product mechanics component explains ingestion outputs", async () => {
"compile into SQL", "compile into SQL",
'"use client"', '"use client"',
"@xyflow/react", "@xyflow/react",
"<ReactFlow", "<FlowCanvas",
"getSmoothStepPath", "getSmoothStepPath",
"animateMotion", "animateMotion",
"mechanics-particle", "mechanics-particle",
@ -97,21 +97,21 @@ test("product mechanics component explains ingestion outputs", async () => {
); );
} }
assert.match( // The ReactFlow canvas config lives in the shared FlowCanvas wrapper, which
component, // product-mechanics renders. Assert the static read-only behavior there.
const flowCanvas = await readDocsFile("components/flow-canvas.tsx");
for (const guard of [
/nodesDraggable=\{false\}/, /nodesDraggable=\{false\}/,
"ReactFlow canvas should disable node dragging", /nodesConnectable=\{false\}/,
);
assert.match(
component,
/panOnDrag=\{false\}/,
"ReactFlow canvas should disable panning",
);
assert.match(
component,
/zoomOnScroll=\{false\}/, /zoomOnScroll=\{false\}/,
"ReactFlow canvas should disable scroll zoom", /elementsSelectable=\{false\}/,
]) {
assert.match(
flowCanvas,
guard,
`shared FlowCanvas should enforce static read-only behavior: ${guard}`,
); );
}
assert.doesNotMatch(component, /raw-sources/); assert.doesNotMatch(component, /raw-sources/);
assert.doesNotMatch(component, /\.ktx/); assert.doesNotMatch(component, /\.ktx/);

View file

@ -0,0 +1,74 @@
import assert from "node:assert/strict";
import { readFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { test } from "node:test";
import { fileURLToPath } from "node:url";
const docsSiteDir = join(dirname(fileURLToPath(import.meta.url)), "..");
async function readDocsFile(path) {
return readFile(join(docsSiteDir, path), "utf8");
}
test("docs introduction renders the serving phase after ingestion", async () => {
const introduction = await readDocsFile(
"content/docs/getting-started/introduction.mdx",
);
assert.match(
introduction,
/import\s+\{\s*ProductRuntime\s*\}\s+from\s+"@\/components\/product-runtime";/,
);
assert.match(introduction, /<ProductRuntime\s*\/>/);
const mechanicsIndex = introduction.indexOf("<ProductMechanics />");
const runtimeIndex = introduction.indexOf("<ProductRuntime />");
const useCaseIndex = introduction.indexOf("## Use it for");
assert.ok(
runtimeIndex > mechanicsIndex,
"serving diagram should appear after the ingestion diagram",
);
assert.ok(
runtimeIndex < useCaseIndex,
"serving diagram should appear before use-case sections",
);
});
test("product runtime component explains the serving cycle", async () => {
const component = await readDocsFile("components/product-runtime.tsx");
for (const expectedText of [
"How serving works",
"Serving flow",
"From an agent request to a governed answer",
"Your agent",
"Claude Code",
"Cursor",
"Codex",
"Search wiki + semantic layer",
"Return approved metrics",
"Compile metrics → SQL",
"Context layer",
"Database",
"search + read",
"read-only",
"wiki/*.md",
"semantic-layer/*.yaml",
'"use client"',
"@xyflow/react",
"FlowCanvas",
"getSmoothStepPath",
"animateMotion",
"runtime-particle",
"buildCyclePath",
]) {
assert.ok(
component.includes(expectedText),
`component should include: ${expectedText}`,
);
}
assert.doesNotMatch(component, /raw-sources/);
assert.doesNotMatch(component, /<img/);
});

View file

@ -1,4 +1,4 @@
# KTX release runbook # ktx release runbook
This runbook covers the maintainer workflow for publishing `@kaelio/ktx` to This runbook covers the maintainer workflow for publishing `@kaelio/ktx` to
npm through GitHub Actions. The workflow uses semantic-release to choose the npm through GitHub Actions. The workflow uses semantic-release to choose the
@ -36,7 +36,7 @@ Before you publish, confirm these requirements:
publish the first stable version as `0.1.0`. publish the first stable version as `0.1.0`.
semantic-release doesn't support choosing an arbitrary first `0.x` stable semantic-release doesn't support choosing an arbitrary first `0.x` stable
release. If KTX has no stable tag yet and you need the first stable release to release. If **ktx** has no stable tag yet and you need the first stable release to
be `0.1.0`, create and push the baseline tag once before running the live be `0.1.0`, create and push the baseline tag once before running the live
stable workflow: stable workflow:
@ -46,7 +46,7 @@ git tag v0.0.0 "${root_commit}"
git push origin v0.0.0 git push origin v0.0.0
``` ```
KTX follows the same versioning schema as the main Kaelio release workflow: **ktx** follows the same versioning schema as the main Kaelio release workflow:
breaking-change and `major` commit markers create a minor release, not an breaking-change and `major` commit markers create a minor release, not an
automatic major release. A major version requires an intentional manual release automatic major release. A major version requires an intentional manual release
path. path.

View file

@ -77,8 +77,6 @@ maintains, validates, and serves that layer.
| Connection ref in prose | **connection id** (lowercase, two words) | "connection ID" | | Connection ref in prose | **connection id** (lowercase, two words) | "connection ID" |
| CLI arg/flag literal | `connectionId` (code font) | — | | CLI arg/flag literal | `connectionId` (code font) | — |
| File path placeholder | `<connection-id>` (code font) | — | | File path placeholder | `<connection-id>` (code font) | — |
| Fast schema mode | **fast ingest** | schema ingest, schema-only ingest |
| AI-enriched mode | **deep ingest** | AI-enriched ingest |
| Ingest of a primary connection | **database ingest** | — | | Ingest of a primary connection | **database ingest** | — |
| Ingest of a context-source connection | **context-source ingest** | bare "source ingest" | | Ingest of a context-source connection | **context-source ingest** | bare "source ingest" |
| Wiki capture | **text ingest** | — | | Wiki capture | **text ingest** | — |

View file

@ -17,7 +17,7 @@ The copied project initializes its own Git repository on first use.
## orbit-relationship-verification ## orbit-relationship-verification
`orbit-relationship-verification/` is a checked-in KTX project used by `orbit-relationship-verification/` is a checked-in **ktx** project used by
`pnpm run relationships:verify-orbit`. It points the `orbit` SQLite connection `pnpm run relationships:verify-orbit`. It points the `orbit` SQLite connection
at the Orbit-style no-declared-constraint relationship fixture and verifies that at the Orbit-style no-declared-constraint relationship fixture and verifies that
relationship enrichment writes nine accepted joins without requiring a local relationship enrichment writes nine accepted joins without requiring a local
@ -27,7 +27,7 @@ warehouse credential.
`postgres-historic/` is a manual Docker-backed smoke for Postgres `postgres-historic/` is a manual Docker-backed smoke for Postgres
query-history ingest via `pg_stat_statements`. It verifies setup, staged query-history ingest via `pg_stat_statements`. It verifies setup, staged
query-history artifacts, KTX daemon batch SQL analysis, bounded pattern query-history artifacts, **ktx** daemon batch SQL analysis, bounded pattern
WorkUnit shards, and no-WorkUnit idempotency for unchanged bucketed table WorkUnit shards, and no-WorkUnit idempotency for unchanged bucketed table
inputs and pattern shards. inputs and pattern shards.

View file

@ -1,6 +1,6 @@
# local-warehouse fixture # local-warehouse fixture
This directory is a contributor fixture for KTX CLI smoke tests. It uses the This directory is a contributor fixture for **ktx** CLI smoke tests. It uses the
internal fake ingest adapter so tests can run without a live database or internal fake ingest adapter so tests can run without a live database or
external service. external service.

View file

@ -5,7 +5,6 @@ storage:
state: sqlite state: sqlite
search: sqlite-fts5 search: sqlite-fts5
git: git:
auto_commit: true
author: "ktx <ktx@example.com>" author: "ktx <ktx@example.com>"
ingest: ingest:
adapters: adapters:
@ -18,5 +17,3 @@ agent:
- sl_query - sl_query
- wiki_search - wiki_search
- sl_read_source - sl_read_source
memory:
auto_commit: true

View file

@ -1,11 +1,11 @@
# Orbit-style relationship discovery verification # Orbit-style relationship discovery verification
This KTX project backs the default `relationships:verify-orbit` command. It uses This **ktx** project backs the default `relationships:verify-orbit` command. It uses
the checked-in Orbit-style SQLite fixture from the relationship discovery the checked-in Orbit-style SQLite fixture from the relationship discovery
benchmark corpus, with no declared primary keys or foreign keys in the database benchmark corpus, with no declared primary keys or foreign keys in the database
schema. schema.
Run from the KTX workspace root: Run from the **ktx** workspace root:
```bash ```bash
pnpm run relationships:verify-orbit pnpm run relationships:verify-orbit

View file

@ -6,7 +6,6 @@ storage:
state: sqlite state: sqlite
search: sqlite-fts5 search: sqlite-fts5
git: git:
auto_commit: true
author: "ktx <ktx@example.com>" author: "ktx <ktx@example.com>"
ingest: ingest:
adapters: [] adapters: []

View file

@ -14,7 +14,7 @@ generated local project.
The managed Python runtime smoke requires `uv` on `PATH`, isolates The managed Python runtime smoke requires `uv` on `PATH`, isolates
`KTX_RUNTIME_ROOT`, verifies `ktx admin runtime status`, runs `ktx sl query --yes` to `KTX_RUNTIME_ROOT`, verifies `ktx admin runtime status`, runs `ktx sl query --yes` to
install the core runtime from the bundled wheel, checks `ktx admin runtime status`, install the core runtime from the bundled wheel, checks `ktx admin runtime status`,
starts and reuses the KTX daemon, and stops it. starts and reuses the **ktx** daemon, and stops it.
The artifact manifest contains the public `@kaelio/ktx` npm tarball and the The artifact manifest contains the public `@kaelio/ktx` npm tarball and the
bundled `kaelio-ktx` runtime wheel. The smoke does not install standalone bundled `kaelio-ktx` runtime wheel. The smoke does not install standalone

View file

@ -17,19 +17,19 @@ unchanged bounded pattern shards do not schedule LLM work.
## Prerequisites ## Prerequisites
- Docker with Compose v2 - Docker with Compose v2
- Node and pnpm matching the KTX workspace - Node and pnpm matching the **ktx** workspace
- `uv` on `PATH` so the KTX-managed Python runtime can install the bundled - `uv` on `PATH` so the **ktx**-managed Python runtime can install the bundled
runtime wheel runtime wheel
## Run ## Run
From the KTX repository root: From the **ktx** repository root:
```bash ```bash
examples/postgres-historic/scripts/smoke.sh examples/postgres-historic/scripts/smoke.sh
``` ```
The smoke creates a temporary KTX project, isolates the managed Python runtime The smoke creates a temporary **ktx** project, isolates the managed Python runtime
under the temporary project parent, starts Postgres on `127.0.0.1:55432`, and under the temporary project parent, starts Postgres on `127.0.0.1:55432`, and
uses this connection URL: uses this connection URL:
@ -41,7 +41,7 @@ Set `KTX_POSTGRES_HISTORIC_KEEP_DOCKER=1` to leave the container running after
the script exits. the script exits.
The smoke validates the query-history raw snapshot path without requiring LLM The smoke validates the query-history raw snapshot path without requiring LLM
credentials. It uses KTX's local stage-only ingest API after `ktx setup`, so the credentials. It uses **ktx**'s local stage-only ingest API after `ktx setup`, so the
deterministic reader, batch SQL parser, stable artifact writer, and diff-based deterministic reader, batch SQL parser, stable artifact writer, and diff-based
WorkUnit planning are checked independently from curation. WorkUnit planning are checked independently from curation.
@ -124,6 +124,6 @@ table.
- Missing grants: confirm `GRANT pg_read_all_stats TO ktx_reader;`. - Missing grants: confirm `GRANT pg_read_all_stats TO ktx_reader;`.
- Empty snapshot: rerun `scripts/generate-workload.sh base` and keep - Empty snapshot: rerun `scripts/generate-workload.sh base` and keep
`--query-history-min-executions 2` for the smoke. `--query-history-min-executions 2` for the smoke.
- SQL-analysis failures: run `pnpm run ktx -- dev runtime status` from the KTX - SQL-analysis failures: run `pnpm run ktx -- dev runtime status` from the **ktx**
repository root and confirm `uv`, the bundled Python wheel, and the managed repository root and confirm `uv`, the bundled Python wheel, and the managed
runtime all pass. runtime all pass.

View file

@ -37,6 +37,9 @@
"@semantic-release/release-notes-generator", "@semantic-release/release-notes-generator",
"conventional-changelog-conventionalcommits" "conventional-changelog-conventionalcommits"
], ],
"ignore": [
".context/**"
],
"ignoreBinaries": [ "ignoreBinaries": [
"uv", "uv",
"lsof" "lsof"

View file

@ -1,10 +1,10 @@
{ {
"name": "ktx-workspace", "name": "ktx-workspace",
"version": "0.7.0", "version": "0.12.0",
"description": "Workspace root for ktx packages", "description": "Workspace root for ktx packages",
"private": true, "private": true,
"type": "module", "type": "module",
"packageManager": "pnpm@11.1.1", "packageManager": "pnpm@11.4.0",
"engines": { "engines": {
"node": ">=22.0.0", "node": ">=22.0.0",
"pnpm": ">=10.20.0" "pnpm": ">=10.20.0"
@ -24,6 +24,7 @@
"dead-code:fix": "biome check . --formatter-enabled=false --assist-enabled=false --write && knip --fix --format", "dead-code:fix": "biome check . --formatter-enabled=false --assist-enabled=false --write && knip --fix --format",
"dead-code:knip": "knip --reporter compact", "dead-code:knip": "knip --reporter compact",
"dead-code:knip:production": "knip --production --reporter compact", "dead-code:knip:production": "knip --production --reporter compact",
"deps:upgrade": "node scripts/upgrade-dependencies.mjs",
"docs": "kill $(lsof -ti:3000) 2>/dev/null; pnpm --filter ktx-docs run dev", "docs": "kill $(lsof -ti:3000) 2>/dev/null; pnpm --filter ktx-docs run dev",
"ktx": "node scripts/run-ktx.mjs", "ktx": "node scripts/run-ktx.mjs",
"link:dev": "node scripts/link-dev-cli.mjs", "link:dev": "node scripts/link-dev-cli.mjs",
@ -31,6 +32,7 @@
"setup:dev": "node scripts/setup-dev.mjs", "setup:dev": "node scripts/setup-dev.mjs",
"release:published-smoke": "node scripts/published-package-smoke.mjs --require-config", "release:published-smoke": "node scripts/published-package-smoke.mjs --require-config",
"release:local-embeddings-smoke": "node scripts/local-embeddings-runtime-smoke.mjs --require-opt-in", "release:local-embeddings-smoke": "node scripts/local-embeddings-runtime-smoke.mjs --require-opt-in",
"release:codex-backend-smoke": "node scripts/codex-backend-live-smoke.mjs",
"release:readiness": "node scripts/release-readiness.mjs", "release:readiness": "node scripts/release-readiness.mjs",
"release:update-version": "node scripts/update-public-release-version.mjs", "release:update-version": "node scripts/update-public-release-version.mjs",
"relationships:acquire-public-fixtures": "node scripts/acquire-public-benchmark-fixtures.mjs", "relationships:acquire-public-fixtures": "node scripts/acquire-public-benchmark-fixtures.mjs",
@ -58,20 +60,15 @@
"@semantic-release/github": "^12.0.8", "@semantic-release/github": "^12.0.8",
"@semantic-release/npm": "^13.1.5", "@semantic-release/npm": "^13.1.5",
"@semantic-release/release-notes-generator": "^14.1.1", "@semantic-release/release-notes-generator": "^14.1.1",
"@types/node": "^25.7.0", "@types/node": "^25.9.1",
"better-sqlite3": "^12.10.0", "better-sqlite3": "^12.10.0",
"conventional-changelog-conventionalcommits": "^9.3.1", "conventional-changelog-conventionalcommits": "^9.3.1",
"knip": "^6.12.2", "knip": "^6.14.1",
"pg": "^8.20.0", "pg": "^8.21.0",
"semantic-release": "^25.0.3", "semantic-release": "^25.0.3",
"typescript": "^6.0.3", "typescript": "^6.0.3",
"yaml": "^2.9.0" "yaml": "^2.9.0"
}, },
"pnpm": {
"onlyBuiltDependencies": [
"better-sqlite3"
]
},
"license": "Apache-2.0", "license": "Apache-2.0",
"repository": { "repository": {
"type": "git", "type": "git",

View file

@ -1,7 +1,11 @@
{ {
"name": "@kaelio/ktx", "name": "@kaelio/ktx",
"version": "0.7.0", "version": "0.12.0",
"description": "Standalone ktx context layer for data agents", "description": "Standalone ktx context layer for data agents",
"author": {
"name": "Kaelio",
"url": "https://www.kaelio.com"
},
"type": "module", "type": "module",
"engines": { "engines": {
"node": ">=22.0.0" "node": ">=22.0.0"
@ -43,36 +47,39 @@
"search:pglite-sl-prototype": "node ../../scripts/pglite-sl-search-prototype.mjs" "search:pglite-sl-prototype": "node ../../scripts/pglite-sl-search-prototype.mjs"
}, },
"dependencies": { "dependencies": {
"@ai-sdk/anthropic": "3.0.77", "@ai-sdk/anthropic": "3.0.78",
"@ai-sdk/devtools": "0.0.17", "@ai-sdk/devtools": "0.0.18",
"@ai-sdk/google-vertex": "^4.0.128", "@ai-sdk/google-vertex": "^4.0.134",
"@anthropic-ai/claude-agent-sdk": "0.3.142", "@anthropic-ai/claude-agent-sdk": "0.3.146",
"@clack/core": "1.3.1",
"@clack/prompts": "1.4.0", "@clack/prompts": "1.4.0",
"@clickhouse/client": "^1.18.4", "@clickhouse/client": "^1.18.5",
"@commander-js/extra-typings": "14.0.0", "@commander-js/extra-typings": "14.0.0",
"@google-cloud/bigquery": "^8.3.1", "@google-cloud/bigquery": "^8.3.1",
"@looker/sdk": "^26.8.0", "@looker/sdk": "^26.8.0",
"@looker/sdk-node": "^26.8.0", "@looker/sdk-node": "^26.8.0",
"@looker/sdk-rtl": "^21.6.5", "@looker/sdk-rtl": "^21.6.5",
"@modelcontextprotocol/sdk": "^1.29.0", "@modelcontextprotocol/sdk": "^1.29.0",
"@notionhq/client": "^5.21.0", "@notionhq/client": "^5.22.0",
"ai": "^6.0.180", "@openai/codex-sdk": "^0.133.0",
"ai": "^6.0.188",
"better-sqlite3": "^12.10.0", "better-sqlite3": "^12.10.0",
"commander": "14.0.3", "commander": "14.0.3",
"fflate": "^0.8.2", "fflate": "^0.8.3",
"handlebars": "^4.7.9", "handlebars": "^4.7.9",
"ink": "^7.0.2", "ink": "^7.0.3",
"lookml-parser": "7.1.0", "lookml-parser": "7.1.0",
"minimatch": "^10.2.5", "minimatch": "^10.2.5",
"mssql": "^12.5.2", "mssql": "^12.5.4",
"mysql2": "^3.22.3", "mysql2": "^3.22.3",
"openai": "^6.37.0", "openai": "^6.38.0",
"p-limit": "^7.3.0", "p-limit": "^7.3.0",
"pg": "^8.20.0", "pg": "^8.21.0",
"posthog-node": "^5.0.0", "posthog-node": "^5.34.9",
"react": "^19.2.6", "react": "^19.2.6",
"semver": "^7.8.1",
"simple-git": "3.36.0", "simple-git": "3.36.0",
"snowflake-sdk": "^2.4.1", "snowflake-sdk": "^2.4.2",
"yaml": "^2.9.0", "yaml": "^2.9.0",
"zod": "^4.4.3" "zod": "^4.4.3"
}, },
@ -81,14 +88,15 @@
"@electric-sql/pglite-socket": "^0.1.5", "@electric-sql/pglite-socket": "^0.1.5",
"@types/better-sqlite3": "^7.6.13", "@types/better-sqlite3": "^7.6.13",
"@types/mssql": "^12.3.0", "@types/mssql": "^12.3.0",
"@types/node": "^25.7.0", "@types/node": "^25.9.1",
"@types/pg": "^8.20.0", "@types/pg": "^8.20.0",
"@types/react": "^19.2.14", "@types/react": "^19.2.15",
"@vitest/coverage-v8": "^4.1.6", "@types/semver": "^7.7.1",
"@vitest/coverage-v8": "^4.1.7",
"ajv": "8.20.0", "ajv": "8.20.0",
"ink-testing-library": "^4.0.0", "ink-testing-library": "^4.0.0",
"typescript": "^6.0.3", "typescript": "^6.0.3",
"vitest": "^4.1.6" "vitest": "^4.1.7"
}, },
"license": "Apache-2.0", "license": "Apache-2.0",
"repository": { "repository": {

View file

@ -24,7 +24,7 @@ export function registerAdminCommands(program: Command, context: KtxCliCommandCo
admin admin
.command('init') .command('init')
.description('Initialize a Git-backed KTX project directory for maintenance scripts') .description('Initialize a Git-backed ktx project directory for maintenance scripts')
.argument('[directory]', 'Project directory') .argument('[directory]', 'Project directory')
.option('--force', 'Rewrite ktx.yaml and scaffold files in an existing project', false) .option('--force', 'Rewrite ktx.yaml and scaffold files in an existing project', false)
.action( .action(

View file

@ -3,6 +3,30 @@ import type { KtxCliIo } from './cli-runtime.js';
const ESC = String.fromCharCode(0x1b); const ESC = String.fromCharCode(0x1b);
export interface CliStyleEnv {
NO_COLOR?: string;
TERM?: string;
}
function ansiEnabled(env: CliStyleEnv = process.env): boolean {
return !env.NO_COLOR && env.TERM !== 'dumb';
}
function ansiColor(text: string, open: number, close: number, env?: CliStyleEnv): string {
if (!ansiEnabled(env)) {
return text;
}
return `${ESC}[${open}m${text}${ESC}[${close}m`;
}
export function dim(text: string, env?: CliStyleEnv): string {
return ansiColor(text, 2, 22, env);
}
export function cyan(text: string, env?: CliStyleEnv): string {
return ansiColor(text, 36, 39, env);
}
export interface RailBufferedSource { export interface RailBufferedSource {
stdoutText(): string; stdoutText(): string;
stderrText(): string; stderrText(): string;
@ -57,27 +81,39 @@ class KtxCliPromptCancelledError extends Error {
} }
export function createClackSpinner(): KtxCliSpinner { export function createClackSpinner(): KtxCliSpinner {
return spinner(); // clack colors the animated spinner frame magenta by default; styleFrame
// (typed in SpinnerOptions, absent from the README) recolors it ktx orange.
return spinner({ styleFrame: orange });
} }
function magenta(text: string): string { // ktx mascot orange (#FF8A4C) via 24-bit truecolor.
return `${ESC}[35m${text}${ESC}[39m`; function orange(text: string): string {
if (!ansiEnabled()) {
return text;
}
return `${ESC}[38;2;255;138;76m${text}${ESC}[39m`;
} }
function red(text: string): string { function red(text: string): string {
return `${ESC}[31m${text}${ESC}[39m`; return ansiColor(text, 31, 39);
} }
/**
* Stderr-only, non-animated spinner. Use this instead of {@link createCliSpinner}
* when the next step reads stdin in raw mode (an Ink TUI or a keypress wait):
* the animated clack spinner seizes stdin via `@clack/core`'s `block()` and
* leaves it dirty, which the following raw-mode reader misreads as a stray key.
*/
export function createStaticCliSpinner(io: KtxCliSpinnerIo): KtxCliSpinner { export function createStaticCliSpinner(io: KtxCliSpinnerIo): KtxCliSpinner {
return { return {
start(message) { start(message) {
io.stderr.write(`${magenta('◐')} ${message}\n`); io.stderr.write(`${orange('◐')} ${message}\n`);
}, },
message(message) { message(message) {
io.stderr.write(`${magenta('│')} ${message}\n`); io.stderr.write(`${orange('│')} ${message}\n`);
}, },
stop(message) { stop(message) {
io.stderr.write(`${magenta('◇')} ${message}\n`); io.stderr.write(`${orange('◇')} ${message}\n`);
}, },
error(message) { error(message) {
io.stderr.write(`${red('■')} ${message}\n`); io.stderr.write(`${red('■')} ${message}\n`);
@ -85,6 +121,30 @@ export function createStaticCliSpinner(io: KtxCliSpinnerIo): KtxCliSpinner {
}; };
} }
/**
* Animated spinner in an interactive terminal, static `◐/◇/■` lines otherwise
* (scripts, CI, piped output) so logs stay clean and uncluttered by frames.
*/
export function createCliSpinner(io: KtxCliIo): KtxCliSpinner {
return io.stdout.isTTY === true ? createClackSpinner() : createStaticCliSpinner(io);
}
export async function runWithCliSpinner<T>(
spinner: KtxCliSpinner,
text: { start: string; success: string; failure: string },
run: () => Promise<T>,
): Promise<T> {
spinner.start(text.start);
try {
const value = await run();
spinner.stop(text.success);
return value;
} catch (error) {
spinner.error(text.failure);
throw error;
}
}
export function createClackPromptAdapter(): KtxCliPromptAdapter { export function createClackPromptAdapter(): KtxCliPromptAdapter {
return { return {
async confirm(options) { async confirm(options) {

View file

@ -21,7 +21,7 @@ export function formatClaudeCodePromptCachingWarning(fields: string[]): string |
if (fields.length === 0) { if (fields.length === 0) {
return null; return null;
} }
return `claude-code ignores ${fields.join(', ')} because the Claude Agent SDK does not expose KTX prompt-cache TTL, tool, or history markers.`; return `claude-code ignores ${fields.join(', ')} because the Claude Agent SDK does not expose ktx prompt-cache TTL, tool, or history markers.`;
} }
export function formatClaudeCodePromptCachingFix(): string { export function formatClaudeCodePromptCachingFix(): string {

View file

@ -2,6 +2,8 @@ import { existsSync } from 'node:fs';
import { join } from 'node:path'; import { join } from 'node:path';
import { Command, type CommandUnknownOpts, InvalidArgumentError } from '@commander-js/extra-typings'; import { Command, type CommandUnknownOpts, InvalidArgumentError } from '@commander-js/extra-typings';
import type { KtxCliDeps, KtxCliIo, KtxCliPackageInfo } from './cli-runtime.js'; import type { KtxCliDeps, KtxCliIo, KtxCliPackageInfo } from './cli-runtime.js';
import { SLACK_HELP_FOOTER, writeErrorCommunityHint } from './community-cta.js';
import { registerCompletionCommands } from './commands/completion-commands.js';
import { registerConnectionCommands } from './commands/connection-commands.js'; import { registerConnectionCommands } from './commands/connection-commands.js';
import { registerIngestCommands } from './commands/ingest-commands.js'; import { registerIngestCommands } from './commands/ingest-commands.js';
import { registerWikiCommands } from './commands/knowledge-commands.js'; import { registerWikiCommands } from './commands/knowledge-commands.js';
@ -15,6 +17,7 @@ import { renderMissingProjectMessage } from './doctor.js';
import { findNearestKtxProjectDir, resolveKtxProjectDir } from './project-resolver.js'; import { findNearestKtxProjectDir, resolveKtxProjectDir } from './project-resolver.js';
import { profileMark, profileSpan } from './startup-profile.js'; import { profileMark, profileSpan } from './startup-profile.js';
import type { CommandOutcome } from './telemetry/index.js'; import type { CommandOutcome } from './telemetry/index.js';
import { prepareUpdateCheckNotice, type PrepareUpdateCheckNoticeOptions } from './update-check/update-check.js';
profileMark('module:cli-program'); profileMark('module:cli-program');
@ -38,6 +41,8 @@ interface KtxCommanderProgramOptions {
runInit: (args: { projectDir: string; force: boolean }, io: KtxCliIo) => Promise<number>; runInit: (args: { projectDir: string; force: boolean }, io: KtxCliIo) => Promise<number>;
} }
type KtxCliUpdateCheckOptions = Pick<PrepareUpdateCheckNoticeOptions, 'env' | 'fetchDistTags' | 'homeDir' | 'now'>;
export interface BuildKtxProgramOptions { export interface BuildKtxProgramOptions {
io: KtxCliIo; io: KtxCliIo;
deps: KtxCliDeps; deps: KtxCliDeps;
@ -46,6 +51,7 @@ export interface BuildKtxProgramOptions {
setExitCode?: (code: number) => void; setExitCode?: (code: number) => void;
argv?: string[]; argv?: string[];
setTelemetryModule?: (telemetry: typeof import('./telemetry/index.js')) => void; setTelemetryModule?: (telemetry: typeof import('./telemetry/index.js')) => void;
updateCheck?: KtxCliUpdateCheckOptions;
} }
type CommanderExitLike = { exitCode: number; code: string; message: string }; type CommanderExitLike = { exitCode: number; code: string; message: string };
@ -246,13 +252,14 @@ export function resolveCommandProjectDirOverride(command: CommandWithGlobalOptio
function createBaseProgram(info: KtxCliPackageInfo, io: KtxCliIo): Command { function createBaseProgram(info: KtxCliPackageInfo, io: KtxCliIo): Command {
return new Command() return new Command()
.name('ktx') .name('ktx')
.description('KTX data agent context layer CLI') .description('ktx data agent context layer CLI')
.option('--project-dir <path>', 'KTX project directory (default: KTX_PROJECT_DIR, nearest ktx.yaml, or cwd)') .option('--project-dir <path>', 'ktx project directory (default: KTX_PROJECT_DIR, nearest ktx.yaml, or cwd)')
.option('--debug', 'Enable diagnostic logging to stderr') .option('--debug', 'Enable diagnostic logging to stderr')
.version(`${info.name} ${info.version}`, '-v, --version', 'Show CLI version') .version(`${info.name} ${info.version}`, '-v, --version', 'Show CLI version')
.helpOption('-h, --help', 'Show this help text') .helpOption('-h, --help', 'Show this help text')
.configureHelp({ showGlobalOptions: true }) .configureHelp({ showGlobalOptions: true })
.showHelpAfterError() .showHelpAfterError()
.addHelpText('after', `\n${SLACK_HELP_FOOTER}`)
.exitOverride() .exitOverride()
.configureOutput({ .configureOutput({
writeOut: (chunk) => io.stdout.write(chunk), writeOut: (chunk) => io.stdout.write(chunk),
@ -430,18 +437,36 @@ export function collectCommandFlagsPresent(command: CommandUnknownOpts): Record<
export function buildKtxProgram(options: BuildKtxProgramOptions): Command { export function buildKtxProgram(options: BuildKtxProgramOptions): Command {
const program = createBaseProgram(options.packageInfo, options.io); const program = createBaseProgram(options.packageInfo, options.io);
let pendingUpdateNotice: string | null = null;
program.hook('preAction', async (_thisCommand, actionCommand) => { program.hook('preAction', async (_thisCommand, actionCommand) => {
// The hidden completion command must stay silent and side-effect free: skip
// the telemetry notice, command span, project checks, and update checks entirely.
if (commandPath(actionCommand as CommandPathNode).includes('__complete')) {
return;
}
const commandNode = actionCommand as CommandPathNode;
const updateCheck = await prepareUpdateCheckNotice({
io: options.io,
env: options.updateCheck?.env,
fetchDistTags: options.updateCheck?.fetchDistTags,
homeDir: options.updateCheck?.homeDir,
installedVersion: options.packageInfo.version,
now: options.updateCheck?.now,
commandOptions: commandOptions(commandNode),
});
pendingUpdateNotice = updateCheck.notice;
const telemetry = await import('./telemetry/index.js'); const telemetry = await import('./telemetry/index.js');
options.setTelemetryModule?.(telemetry); options.setTelemetryModule?.(telemetry);
await telemetry.showTelemetryNoticeIfNeeded(options.io, options.packageInfo); await telemetry.showTelemetryNoticeIfNeeded(options.io, options.packageInfo);
const commandNode = actionCommand as CommandPathNode;
const path = commandPath(commandNode); const path = commandPath(commandNode);
const projectDir = resolveCommandProjectDir(commandNode); const projectDir = resolveCommandProjectDir(commandNode);
const hasProject = ktxYamlExists(projectDir); const hasProject = ktxYamlExists(projectDir);
const attachProjectGroup = shouldAttachCommandProjectGroup(path, hasProject); const attachProjectGroup = shouldAttachCommandProjectGroup(path, hasProject);
telemetry.beginCommandSpan({ telemetry.beginCommandSpan({
commandPath: path, commandPath: path,
flagsPresent: collectCommandFlagsPresent(commandNode as unknown as CommandUnknownOpts), flagsPresent: collectCommandFlagsPresent(actionCommand),
projectDir: attachProjectGroup ? projectDir : undefined, projectDir: attachProjectGroup ? projectDir : undefined,
hasProject, hasProject,
attachProjectGroup, attachProjectGroup,
@ -451,6 +476,13 @@ export function buildKtxProgram(options: BuildKtxProgramOptions): Command {
ensureProjectAvailable(options.io, commandNode); ensureProjectAvailable(options.io, commandNode);
}); });
program.hook('postAction', () => {
if (pendingUpdateNotice) {
options.io.stderr.write(pendingUpdateNotice);
pendingUpdateNotice = null;
}
});
const context: KtxCliCommandContext = { const context: KtxCliCommandContext = {
io: options.io, io: options.io,
deps: options.deps, deps: options.deps,
@ -476,6 +508,7 @@ export function buildKtxProgram(options: BuildKtxProgramOptions): Command {
registerStatusCommands(program, context); registerStatusCommands(program, context);
registerMcpCommands(program, context); registerMcpCommands(program, context);
registerAdminCommands(program, context); registerAdminCommands(program, context);
registerCompletionCommands(program, context);
return program; return program;
} }
@ -522,7 +555,15 @@ export async function runCommanderKtxCli(
try { try {
return await runBareInteractiveCommand(program, io, context); return await runBareInteractiveCommand(program, io, context);
} catch (error) { } catch (error) {
const telemetry = await import('./telemetry/index.js');
await telemetry.reportException({
error,
context: { source: 'bare-interactive', handled: true, fatal: false },
packageInfo: info,
io,
});
io.stderr.write(`${formatCliError(error)}\n`); io.stderr.write(`${formatCliError(error)}\n`);
writeErrorCommunityHint(io, 'error');
return 1; return 1;
} }
} }
@ -547,6 +588,7 @@ export async function runCommanderKtxCli(
exitCode = error.exitCode === 0 ? 0 : 1; exitCode = error.exitCode === 0 ? 0 : 1;
} else { } else {
io.stderr.write(`${formatCliError(error)}\n`); io.stderr.write(`${formatCliError(error)}\n`);
writeErrorCommunityHint(io, 'error');
exitCode = 1; exitCode = 1;
} }
} finally { } finally {
@ -556,6 +598,23 @@ export async function runCommanderKtxCli(
outcome: commandOutcomeForParseResult(parseError, exitCode), outcome: commandOutcomeForParseResult(parseError, exitCode),
error: parseError, error: parseError,
}); });
if (
parseError &&
!isCommanderExit(parseError) &&
!isKtxProjectMissingAbortError(parseError)
) {
await telemetryModule.reportException({
error: parseError,
context: {
source: completed?.commandPath.join(' ') ?? 'commander parseAsync',
handled: true,
fatal: false,
},
projectDir: completed?.projectGroupAttached ? completed.projectDir : undefined,
packageInfo: info,
io,
});
}
await telemetryModule.emitCompletedCommand({ completed, packageInfo: info, io }); await telemetryModule.emitCompletedCommand({ completed, packageInfo: info, io });
await telemetryModule.shutdownTelemetryEmitter(); await telemetryModule.shutdownTelemetryEmitter();
} }

View file

@ -12,6 +12,7 @@ import type { KtxSqlArgs } from './sql.js';
import { profileMark, profileSpan } from './startup-profile.js'; import { profileMark, profileSpan } from './startup-profile.js';
import type { KtxTextIngestArgs } from './text-ingest.js'; import type { KtxTextIngestArgs } from './text-ingest.js';
import { assertCliVersion } from './release-version.js'; import { assertCliVersion } from './release-version.js';
import { writeErrorCommunityHint } from './community-cta.js';
profileMark('module:cli-runtime'); profileMark('module:cli-runtime');
@ -60,7 +61,7 @@ export function packageInfoFromJson(packageJson: unknown): KtxCliPackageInfo {
typeof packageJson.name !== 'string' || typeof packageJson.name !== 'string' ||
typeof packageJson.version !== 'string' typeof packageJson.version !== 'string'
) { ) {
throw new Error('Invalid KTX CLI package metadata'); throw new Error('Invalid ktx CLI package metadata');
} }
return { return {
@ -76,7 +77,7 @@ async function runInit(args: { projectDir: string; force: boolean }, io: KtxCliI
force: args.force, force: args.force,
}); });
io.stdout.write(`Initialized KTX project at ${result.projectDir}\n`); io.stdout.write(`Initialized ktx project at ${result.projectDir}\n`);
io.stdout.write(`Config: ${result.configPath}\n`); io.stdout.write(`Config: ${result.configPath}\n`);
io.stdout.write(`Commit: ${result.commitHash ?? 'none'}\n`); io.stdout.write(`Commit: ${result.commitHash ?? 'none'}\n`);
return 0; return 0;
@ -89,6 +90,94 @@ export async function runInitForCommander(
return await runInit(args, io); return await runInit(args, io);
} }
function signalExitCode(signal: NodeJS.Signals): number {
// 128 + signal number: SIGINT (2) -> 130, SIGTERM (15) -> 143.
return signal === 'SIGTERM' ? 143 : 130;
}
/**
* Flush telemetry on interrupt for the real CLI process. `capture()` is
* fire-and-forget and the only flush guarantee lives in a `finally` a signal
* skips, so Ctrl-C / `kill` of a long-running command (ingest, `mcp stdio`)
* would otherwise drop its `command` event and queued events. Installed only
* when driving the actual process; programmatic/test callers pass their own
* `io` and never reach here. Returns a disposer that removes the listeners.
*/
function installTelemetrySignalFlush(io: KtxCliIo, info: KtxCliPackageInfo): () => void {
let handling = false;
const handle = (signal: NodeJS.Signals): void => {
if (handling) {
process.exit(signalExitCode(signal));
}
handling = true;
void (async () => {
try {
const { emitAbortedCommandAndShutdown } = await import('./telemetry/index.js');
await emitAbortedCommandAndShutdown({ packageInfo: info, io });
} catch {
// Best-effort: never let a telemetry hiccup block the interrupt exit.
}
process.exit(signalExitCode(signal));
})();
};
const onSigint = (): void => handle('SIGINT');
const onSigterm = (): void => handle('SIGTERM');
process.on('SIGINT', onSigint);
process.on('SIGTERM', onSigterm);
return () => {
process.off('SIGINT', onSigint);
process.off('SIGTERM', onSigterm);
};
}
/** @internal */
export function createGlobalExceptionReporter(io: KtxCliIo, info: KtxCliPackageInfo) {
return async (source: 'uncaughtException' | 'unhandledRejection', error: unknown): Promise<void> => {
const { reportException, shutdownTelemetryEmitter } = await import('./telemetry/index.js');
await reportException({
error,
context: { source, handled: false, fatal: true },
io,
packageInfo: info,
immediate: true,
});
await shutdownTelemetryEmitter();
};
}
/** @internal */
export function writeGlobalExceptionToStderr(io: KtxCliIo, error: unknown): void {
if (error instanceof Error && error.stack) {
io.stderr.write(`${error.stack}\n`);
} else {
io.stderr.write(`${String(error)}\n`);
}
writeErrorCommunityHint(io, 'crash');
}
export function installGlobalExceptionHandlers(io: KtxCliIo, info: KtxCliPackageInfo): () => void {
const report = createGlobalExceptionReporter(io, info);
const handle = (source: 'uncaughtException' | 'unhandledRejection', error: unknown): void => {
void (async () => {
try {
await report(source, error);
} catch {
// Best-effort: preserve Node's process termination behavior.
}
writeGlobalExceptionToStderr(io, error);
process.exit(1);
})();
};
const onUncaught = (error: Error): void => handle('uncaughtException', error);
const onUnhandled = (reason: unknown): void => handle('unhandledRejection', reason);
process.on('uncaughtException', onUncaught);
process.on('unhandledRejection', onUnhandled);
return () => {
process.off('uncaughtException', onUncaught);
process.off('unhandledRejection', onUnhandled);
};
}
export async function runKtxCli( export async function runKtxCli(
argv = process.argv.slice(2), argv = process.argv.slice(2),
io: KtxCliIo = process, io: KtxCliIo = process,
@ -98,7 +187,17 @@ export async function runKtxCli(
profileMark('runtime:runKtxCli'); profileMark('runtime:runKtxCli');
const { runCommanderKtxCli } = await profileSpan('import ./cli-program.js', () => import('./cli-program.js')); const { runCommanderKtxCli } = await profileSpan('import ./cli-program.js', () => import('./cli-program.js'));
// Real-process entry only: flush telemetry if interrupted. Test/programmatic
// callers pass their own `io`, so they never install process-level handlers.
const removeSignalFlush = (io as unknown) === process ? installTelemetrySignalFlush(io, info) : undefined;
const removeGlobalExceptionHandlers =
(io as unknown) === process ? installGlobalExceptionHandlers(io, info) : undefined;
try {
return await runCommanderKtxCli(argv, io, deps, info, { return await runCommanderKtxCli(argv, io, deps, info, {
runInit: runInitForCommander, runInit: runInitForCommander,
}); });
} finally {
removeGlobalExceptionHandlers?.();
removeSignalFlush?.();
}
} }

View file

@ -16,7 +16,11 @@ export function walkCommandTree(command: CommandUnknownOpts): CommandTreeNode {
description: command.description(), description: command.description(),
aliases: command.aliases(), aliases: command.aliases(),
arguments: command.registeredArguments.map(formatArgumentDeclaration), arguments: command.registeredArguments.map(formatArgumentDeclaration),
children: command.commands.map((child) => walkCommandTree(child)), // Internal commands (e.g. the shell-completion helper `__complete`) use a
// `__` prefix and are omitted from the human-facing command tree.
children: command.commands
.filter((child) => !child.name().startsWith('__'))
.map((child) => walkCommandTree(child)),
}; };
} }

View file

@ -0,0 +1,44 @@
import { Argument, type Command } from '@commander-js/extra-typings';
import type { KtxCliCommandContext } from '../cli-program.js';
import { computeCompletions } from '../completion/complete-engine.js';
import { completionScript } from '../completion/completion-scripts.js';
import { createProjectCompletionProviders } from '../completion/dynamic-candidates.js';
import { profileMark } from '../startup-profile.js';
profileMark('module:commands/completion-commands');
export function registerCompletionCommands(program: Command, context: KtxCliCommandContext): void {
program
.command('completion')
.description('Print a shell completion script for ktx')
.addArgument(new Argument('<shell>', 'Target shell').choices(['zsh', 'bash']))
.addHelpText(
'after',
'\nEnable completion by adding the matching line to your shell startup file:\n' +
' zsh: eval "$(ktx completion zsh)"\n' +
' bash: eval "$(ktx completion bash)"\n',
)
.action((shell) => {
context.io.stdout.write(completionScript(shell));
});
// Hidden command invoked by the generated shell scripts. It must only ever
// print newline-separated candidates to stdout and exit 0, so a TAB press is
// never disrupted by an error, a telemetry notice, or a parse failure.
program
.command('__complete', { hidden: true })
.argument('[words...]')
.allowUnknownOption(true)
.helpOption(false)
.action(async (words: string[]) => {
try {
const candidates = await computeCompletions(program, words, createProjectCompletionProviders());
if (candidates.length > 0) {
context.io.stdout.write(`${candidates.join('\n')}\n`);
}
} catch {
// Swallow: completion must never break the shell.
}
context.setExitCode(0);
});
}

View file

@ -37,7 +37,7 @@ export function registerConnectionCommands(program: Command, context: KtxCliComm
connection connection
.command('test') .command('test')
.description('Test one or all configured connections (default: all)') .description('Test one or all configured connections (default: all)')
.argument('[connectionId]', 'KTX connection id to test (omit to test all)') .argument('[connectionId]', 'ktx connection id to test (omit to test all)')
.option('--all', 'Test every configured connection and print a summary list') .option('--all', 'Test every configured connection and print a summary list')
.action(async (connectionId: string | undefined, options: { all?: boolean }, command) => { .action(async (connectionId: string | undefined, options: { all?: boolean }, command) => {
if (options.all === true && connectionId !== undefined) { if (options.all === true && connectionId !== undefined) {

View file

@ -25,18 +25,16 @@ export function registerIngestCommands(
): void { ): void {
const ingest = program const ingest = program
.command('ingest') .command('ingest')
.description('Build or inspect KTX context, or capture text into memory') .description('Build or inspect ktx context, or capture text into memory')
.usage('[options] [connectionId]') .usage('[options] [connectionId]')
.argument('[connectionId]', 'Configured connection id to ingest (omit to ingest all)') .argument('[connectionId]', 'Configured connection id to ingest (omit to ingest all)')
.option('--all', 'Ingest all configured connections', false) .option('--all', 'Ingest all configured connections', false)
.addOption(new Option('--fast', 'Use deterministic database schema ingest').conflicts('deep'))
.addOption(new Option('--deep', 'Use AI-enriched database ingest').conflicts('fast'))
.addOption(new Option('--query-history', 'Include database query-history usage patterns').conflicts('noQueryHistory')) .addOption(new Option('--query-history', 'Include database query-history usage patterns').conflicts('noQueryHistory'))
.addOption(new Option('--no-query-history', 'Skip database query-history usage patterns')) .addOption(new Option('--no-query-history', 'Skip database query-history usage patterns'))
.option('--query-history-window-days <days>', 'Query-history lookback window for this run', parsePositiveIntegerOption) .option('--query-history-window-days <days>', 'Query-history lookback window for this run', parsePositiveIntegerOption)
.option('--text <content>', 'Capture inline text into KTX memory; repeatable', collectOption, []) .option('--text <content>', 'Capture inline text into ktx memory; repeatable', collectOption, [])
.option('--file <path>', 'Capture a text file into KTX memory; use - for stdin; repeatable', collectOption, []) .option('--file <path>', 'Capture a text file into ktx memory; use - for stdin; repeatable', collectOption, [])
.option('--connection-id <connectionId>', 'KTX connection id to tag captured text/file notes') .option('--connection-id <connectionId>', 'ktx connection id to tag captured text/file notes')
.option('--user-id <id>', 'Memory user id for text/file capture attribution', 'local-cli') .option('--user-id <id>', 'Memory user id for text/file capture attribution', 'local-cli')
.option('--fail-fast', 'Stop after the first failed text/file item', false) .option('--fail-fast', 'Stop after the first failed text/file item', false)
.addOption(new Option('--plain', 'Print plain text output').conflicts(['json'])) .addOption(new Option('--plain', 'Print plain text output').conflicts(['json']))
@ -87,8 +85,6 @@ export function registerIngestCommands(
all: selection.kind === 'all', all: selection.kind === 'all',
json: options.json === true, json: options.json === true,
inputMode: options.input === false ? 'disabled' : 'auto', inputMode: options.input === false ? 'disabled' : 'auto',
...(options.fast === true ? { depth: 'fast' as const } : {}),
...(options.deep === true ? { depth: 'deep' as const } : {}),
queryHistory, queryHistory,
...(options.queryHistoryWindowDays !== undefined ? { queryHistoryWindowDays: options.queryHistoryWindowDays } : {}), ...(options.queryHistoryWindowDays !== undefined ? { queryHistoryWindowDays: options.queryHistoryWindowDays } : {}),
cliVersion: context.packageInfo.version, cliVersion: context.packageInfo.version,

View file

@ -21,9 +21,9 @@ function isDebugEnabled(command: CommandWithGlobalOptions): boolean {
} }
export function registerWikiCommands(program: Command, context: KtxCliCommandContext): void { export function registerWikiCommands(program: Command, context: KtxCliCommandContext): void {
program const wiki = program
.command('wiki') .command('wiki')
.description('List or search local wiki pages') .description('List, search, or read local wiki pages')
.usage('[options] [query...]') .usage('[options] [query...]')
.argument('[query...]', 'Search query; omit to list all pages') .argument('[query...]', 'Search query; omit to list all pages')
.option('--user-id <id>', 'Local user id', 'local') .option('--user-id <id>', 'Local user id', 'local')
@ -76,4 +76,18 @@ export function registerWikiCommands(program: Command, context: KtxCliCommandCon
}); });
}, },
); );
wiki
.command('read')
.description('Read a wiki page file by key')
.argument('<key>', 'Wiki page key')
.action(async (key: string, _options, command) => {
const parentOpts = command.parent?.opts() as { userId?: string } | undefined;
await runKnowledgeArgs(context, {
command: 'read',
projectDir: resolveCommandProjectDir(command),
key,
userId: parentOpts?.userId ?? 'local',
});
});
} }

View file

@ -27,11 +27,11 @@ function binPath(): string {
function formatMcpStartResultMessage(input: { status: 'started' | 'already-running'; url: string }): string { function formatMcpStartResultMessage(input: { status: 'started' | 'already-running'; url: string }): string {
return [ return [
input.status === 'started' ? `KTX MCP daemon started: ${input.url}` : `KTX MCP daemon already running: ${input.url}`, input.status === 'started' ? `ktx MCP daemon started: ${input.url}` : `ktx MCP daemon already running: ${input.url}`,
'', '',
'KTX is ready for configured agents.', 'ktx is ready for configured agents.',
'Open your agent for this KTX project and ask a data question, for example:', 'Open your agent for this ktx project and ask a data question, for example:',
' "Use KTX to show me the available tables and metrics."', ' "Use ktx to show me the available tables and metrics."',
'', '',
].join('\n'); ].join('\n');
} }
@ -50,14 +50,14 @@ async function printMcpStatus(context: KtxCliCommandContext, projectDir: string)
export function registerMcpCommands(program: Command, context: KtxCliCommandContext): void { export function registerMcpCommands(program: Command, context: KtxCliCommandContext): void {
const mcp = program const mcp = program
.command('mcp') .command('mcp')
.description('Manage the KTX MCP HTTP server (bare command: show status)') .description('Manage the ktx MCP HTTP server (bare command: show status)')
.action(async (_options, command) => { .action(async (_options, command) => {
await printMcpStatus(context, resolveCommandProjectDir(command)); await printMcpStatus(context, resolveCommandProjectDir(command));
}); });
mcp mcp
.command('stdio') .command('stdio')
.description('Run the KTX MCP server over stdio') .description('Run the ktx MCP server over stdio')
.action(async (_options, command) => { .action(async (_options, command) => {
await (context.deps.mcp?.runStdioServer ?? runKtxMcpStdioServer)({ await (context.deps.mcp?.runStdioServer ?? runKtxMcpStdioServer)({
projectDir: resolveCommandProjectDir(command), projectDir: resolveCommandProjectDir(command),
@ -68,7 +68,7 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
mcp mcp
.command('start') .command('start')
.description('Start the KTX MCP HTTP server') .description('Start the ktx MCP HTTP server')
.option('--host <host>', 'Host to bind', '127.0.0.1') .option('--host <host>', 'Host to bind', '127.0.0.1')
.option('--port <n>', 'Port to bind', parsePositiveIntegerOption, 7878) .option('--port <n>', 'Port to bind', parsePositiveIntegerOption, 7878)
.option('--token <token>', 'Bearer token required for non-loopback binding') .option('--token <token>', 'Bearer token required for non-loopback binding')
@ -96,7 +96,7 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
allowedOrigins: options.allowedOrigin, allowedOrigins: options.allowedOrigin,
io: context.io, io: context.io,
}); });
context.io.stdout.write(`KTX MCP server listening at http://${options.host}:${options.port}/mcp\n`); context.io.stdout.write(`ktx MCP server listening at http://${options.host}:${options.port}/mcp\n`);
return; return;
} }
const result = await (context.deps.mcp?.startDaemon ?? startKtxMcpDaemon)({ const result = await (context.deps.mcp?.startDaemon ?? startKtxMcpDaemon)({
@ -114,24 +114,24 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
mcp mcp
.command('stop') .command('stop')
.description('Stop the KTX MCP daemon') .description('Stop the ktx MCP daemon')
.action(async (_options, command) => { .action(async (_options, command) => {
const result = await (context.deps.mcp?.stopDaemon ?? stopKtxMcpDaemon)({ const result = await (context.deps.mcp?.stopDaemon ?? stopKtxMcpDaemon)({
projectDir: resolveCommandProjectDir(command), projectDir: resolveCommandProjectDir(command),
}); });
context.io.stdout.write(result.status === 'stopped' ? 'KTX MCP daemon stopped.\n' : 'KTX MCP daemon is not running.\n'); context.io.stdout.write(result.status === 'stopped' ? 'ktx MCP daemon stopped.\n' : 'ktx MCP daemon is not running.\n');
}); });
mcp mcp
.command('status') .command('status')
.description('Show KTX MCP daemon status') .description('Show ktx MCP daemon status')
.action(async (_options, command) => { .action(async (_options, command) => {
await printMcpStatus(context, resolveCommandProjectDir(command)); await printMcpStatus(context, resolveCommandProjectDir(command));
}); });
mcp mcp
.command('logs') .command('logs')
.description('Print the KTX MCP daemon log') .description('Print the ktx MCP daemon log')
.option('--follow', 'Follow log output', false) .option('--follow', 'Follow log output', false)
.action(async (options, command) => { .action(async (options, command) => {
const logPath = mcpDaemonLayout(resolveCommandProjectDir(command)).logPath; const logPath = mcpDaemonLayout(resolveCommandProjectDir(command)).logPath;

View file

@ -18,7 +18,7 @@ async function runRuntimeArgs(context: KtxCliCommandContext, args: KtxRuntimeArg
export function registerRuntimeCommands(program: Command, context: KtxCliCommandContext): void { export function registerRuntimeCommands(program: Command, context: KtxCliCommandContext): void {
const runtime = program const runtime = program
.command('runtime') .command('runtime')
.description('Install, start, stop, and inspect the KTX-managed Python runtime') .description('Install, start, stop, and inspect the ktx-managed Python runtime')
.showHelpAfterError(); .showHelpAfterError();
runtime runtime
@ -38,7 +38,7 @@ export function registerRuntimeCommands(program: Command, context: KtxCliCommand
runtime runtime
.command('start') .command('start')
.description('Start the KTX daemon') .description('Start the ktx daemon')
.addOption(createRuntimeFeatureOption()) .addOption(createRuntimeFeatureOption())
.option('--force', 'Restart even when a matching daemon is already running', false) .option('--force', 'Restart even when a matching daemon is already running', false)
.action(async (options: { feature: RuntimeFeature; force?: boolean }, command: CommandWithGlobalOptions) => { .action(async (options: { feature: RuntimeFeature; force?: boolean }, command: CommandWithGlobalOptions) => {
@ -53,8 +53,8 @@ export function registerRuntimeCommands(program: Command, context: KtxCliCommand
runtime runtime
.command('stop') .command('stop')
.description('Stop the KTX daemon') .description('Stop the ktx daemon')
.option('--all', 'Stop all KTX daemon processes recorded or discoverable on this machine', false) .option('--all', 'Stop all ktx daemon processes recorded or discoverable on this machine', false)
.action(async (options: { all?: boolean }, command: CommandWithGlobalOptions) => { .action(async (options: { all?: boolean }, command: CommandWithGlobalOptions) => {
await runRuntimeArgs(context, { await runRuntimeArgs(context, {
command: 'stop', command: 'stop',

View file

@ -2,7 +2,7 @@ import { type Command, InvalidArgumentError, Option } from '@commander-js/extra-
import type { KtxCliCommandContext } from '../cli-program.js'; import type { KtxCliCommandContext } from '../cli-program.js';
import { resolveCommandProjectDir } from '../cli-program.js'; import { resolveCommandProjectDir } from '../cli-program.js';
import type { KtxSetupDatabaseDriver } from '../setup-databases.js'; import type { KtxSetupDatabaseDriver } from '../setup-databases.js';
import type { KtxSetupLlmBackend } from '../setup-models.js'; import { isKtxSetupLlmBackend, type KtxSetupLlmBackend } from '../setup-models.js';
import type { KtxSetupSourceType } from '../setup-sources.js'; import type { KtxSetupSourceType } from '../setup-sources.js';
async function runSetupArgs( async function runSetupArgs(
@ -16,7 +16,7 @@ async function runSetupArgs(
function positiveInteger(value: string): number { function positiveInteger(value: string): number {
const parsed = Number.parseInt(value, 10); const parsed = Number.parseInt(value, 10);
if (!Number.isInteger(parsed) || parsed <= 0) { if (!Number.isInteger(parsed) || parsed <= 0) {
throw new Error(`Expected a positive integer, received ${value}`); throw new InvalidArgumentError(`Expected a positive integer, received ${value}`);
} }
return parsed; return parsed;
} }
@ -29,7 +29,7 @@ function embeddingBackend(value: string): 'openai' | 'sentence-transformers' {
} }
function llmBackend(value: string): KtxSetupLlmBackend { function llmBackend(value: string): KtxSetupLlmBackend {
if (value === 'anthropic' || value === 'vertex' || value === 'claude-code') { if (isKtxSetupLlmBackend(value)) {
return value; return value;
} }
throw new InvalidArgumentError(`invalid choice '${value}'`); throw new InvalidArgumentError(`invalid choice '${value}'`);
@ -89,13 +89,13 @@ function shouldShowSetupEntryMenu(
target?: string; target?: string;
global?: boolean; global?: boolean;
local?: boolean; local?: boolean;
installDir?: string;
skipAgents?: boolean; skipAgents?: boolean;
yes?: boolean; yes?: boolean;
input?: boolean; input?: boolean;
llmBackend?: KtxSetupLlmBackend; llmBackend?: KtxSetupLlmBackend;
anthropicApiKeyEnv?: string; anthropicApiKeyEnv?: string;
anthropicApiKeyFile?: string; anthropicApiKeyFile?: string;
llmModel?: string;
vertexProject?: string; vertexProject?: string;
vertexLocation?: string; vertexLocation?: string;
skipLlm?: boolean; skipLlm?: boolean;
@ -160,13 +160,13 @@ function shouldShowSetupEntryMenu(
'target', 'target',
'global', 'global',
'local', 'local',
'installDir',
'skipAgents', 'skipAgents',
'yes', 'yes',
'input', 'input',
'llmBackend', 'llmBackend',
'anthropicApiKeyEnv', 'anthropicApiKeyEnv',
'anthropicApiKeyFile', 'anthropicApiKeyFile',
'llmModel',
'vertexProject', 'vertexProject',
'vertexLocation', 'vertexLocation',
'skipLlm', 'skipLlm',
@ -204,8 +204,8 @@ function shouldShowSetupEntryMenu(
export function registerSetupCommands(program: Command, context: KtxCliCommandContext): void { export function registerSetupCommands(program: Command, context: KtxCliCommandContext): void {
const setup = program const setup = program
.command('setup') .command('setup')
.description('Set up or resume a local KTX project') .description('Set up or resume a local ktx project')
.addOption(new Option('--project-dir <path>', 'KTX project directory').hideHelp()) .addOption(new Option('--project-dir <path>', 'ktx project directory').hideHelp())
.option('--agents', 'Install agent integration only', false) .option('--agents', 'Install agent integration only', false)
.addOption( .addOption(
new Option('--target <target>', 'Agent target').choices([ new Option('--target <target>', 'Agent target').choices([
@ -219,6 +219,10 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
) )
.option('--global', 'Install agent integration into the global target scope', false) .option('--global', 'Install agent integration into the global target scope', false)
.option('--local', 'Install Claude Code MCP config into the private per-project ~/.claude.json scope', false) .option('--local', 'Install Claude Code MCP config into the private per-project ~/.claude.json scope', false)
.option(
'--install-dir <path>',
'Directory to install project-scoped agent config into (defaults to the ktx project directory)',
)
.addOption(new Option('--skip-agents', 'Leave agent integration incomplete for now').hideHelp().default(false)) .addOption(new Option('--skip-agents', 'Leave agent integration incomplete for now').hideHelp().default(false))
.option('--yes', 'Accept project creation and runtime install defaults where setup confirms', false) .option('--yes', 'Accept project creation and runtime install defaults where setup confirms', false)
.option('--no-input', 'Disable interactive terminal input') .option('--no-input', 'Disable interactive terminal input')
@ -229,7 +233,6 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
.addOption( .addOption(
new Option('--anthropic-api-key-file <path>', 'File containing the Anthropic API key').hideHelp(), new Option('--anthropic-api-key-file <path>', 'File containing the Anthropic API key').hideHelp(),
) )
.addOption(new Option('--llm-model <model>', 'LLM model ID or backend model alias').hideHelp())
.addOption(new Option('--vertex-project <project>', 'Google Vertex AI project ID, env:NAME, or file:/path').hideHelp()) .addOption(new Option('--vertex-project <project>', 'Google Vertex AI project ID, env:NAME, or file:/path').hideHelp())
.addOption(new Option('--vertex-location <location>', 'Google Vertex AI location, env:NAME, or file:/path').hideHelp()) .addOption(new Option('--vertex-location <location>', 'Google Vertex AI location, env:NAME, or file:/path').hideHelp())
.addOption(new Option('--skip-llm', 'Leave LLM setup incomplete for now').hideHelp().default(false)) .addOption(new Option('--skip-llm', 'Leave LLM setup incomplete for now').hideHelp().default(false))
@ -298,7 +301,7 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
.hideHelp(), .hideHelp(),
) )
.addOption( .addOption(
new Option('--skip-databases', 'Leave database setup incomplete; KTX cannot work until a database is added') new Option('--skip-databases', 'Leave database setup incomplete; ktx cannot work until a database is added')
.hideHelp() .hideHelp()
.default(false), .default(false),
) )
@ -308,9 +311,14 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
.addOption(new Option('--source-git-url <url>', 'Git URL for dbt, MetricFlow, or LookML').hideHelp()) .addOption(new Option('--source-git-url <url>', 'Git URL for dbt, MetricFlow, or LookML').hideHelp())
.addOption(new Option('--source-branch <branch>', 'Git branch for source setup').hideHelp()) .addOption(new Option('--source-branch <branch>', 'Git branch for source setup').hideHelp())
.addOption(new Option('--source-subpath <path>', 'Repo subpath for source setup').hideHelp()) .addOption(new Option('--source-subpath <path>', 'Repo subpath for source setup').hideHelp())
.addOption(new Option('--source-auth-token-ref <ref>', 'env: or file: credential ref for source repo auth').hideHelp()) .addOption(
new Option(
'--source-auth-token-ref <ref>',
'env: or file: credential ref for source repo auth or Notion integration token',
).hideHelp(),
)
.addOption(new Option('--source-url <url>', 'Source service URL for Metabase or Looker').hideHelp()) .addOption(new Option('--source-url <url>', 'Source service URL for Metabase or Looker').hideHelp())
.addOption(new Option('--source-api-key-ref <ref>', 'env: or file: API key ref for Metabase or Notion').hideHelp()) .addOption(new Option('--source-api-key-ref <ref>', 'env: or file: API key ref for Metabase').hideHelp())
.addOption(new Option('--source-client-id <id>', 'Looker client id').hideHelp()) .addOption(new Option('--source-client-id <id>', 'Looker client id').hideHelp())
.addOption(new Option('--source-client-secret-ref <ref>', 'env: or file: Looker client secret ref').hideHelp()) .addOption(new Option('--source-client-secret-ref <ref>', 'env: or file: Looker client secret ref').hideHelp())
.addOption(new Option('--source-warehouse-connection-id <id>', 'Mapped warehouse connection id').hideHelp()) .addOption(new Option('--source-warehouse-connection-id <id>', 'Mapped warehouse connection id').hideHelp())
@ -392,6 +400,16 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
context.setExitCode(1); context.setExitCode(1);
return; return;
} }
if (options.installDir && (options.global || options.local)) {
context.io.stderr.write('Choose either --install-dir or a scope flag (--global / --local), not both.\n');
context.setExitCode(1);
return;
}
if (options.installDir && options.target === 'claude-desktop') {
context.io.stderr.write('--install-dir does not apply to --target claude-desktop, which is always global.\n');
context.setExitCode(1);
return;
}
const creatingDatabaseConnection = options.database.length > 0 || options.databaseUrl !== undefined; const creatingDatabaseConnection = options.database.length > 0 || options.databaseUrl !== undefined;
if (creatingDatabaseConnection && options.databaseConnectionId.length > 1) { if (creatingDatabaseConnection && options.databaseConnectionId.length > 1) {
@ -401,6 +419,8 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
} }
const resolvedAgentScope = options.local ? 'local' : options.global ? 'global' : 'project'; const resolvedAgentScope = options.local ? 'local' : options.global ? 'global' : 'project';
const debugEnabled =
((command.optsWithGlobals ? command.optsWithGlobals() : command.opts()) as { debug?: unknown }).debug === true;
await runSetupArgs(context, { await runSetupArgs(context, {
command: 'run', command: 'run',
projectDir: resolveCommandProjectDir(command), projectDir: resolveCommandProjectDir(command),
@ -408,14 +428,15 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
agents: options.agents === true, agents: options.agents === true,
...(options.target ? { target: options.target } : {}), ...(options.target ? { target: options.target } : {}),
agentScope: resolvedAgentScope, agentScope: resolvedAgentScope,
...(options.installDir ? { installRoot: options.installDir } : {}),
skipAgents: options.skipAgents === true, skipAgents: options.skipAgents === true,
inputMode: options.input === false ? 'disabled' : 'auto', inputMode: options.input === false ? 'disabled' : 'auto',
...(debugEnabled ? { debug: true } : {}),
yes: options.yes === true, yes: options.yes === true,
cliVersion: context.packageInfo.version, cliVersion: context.packageInfo.version,
...(options.llmBackend ? { llmBackend: options.llmBackend } : {}), ...(options.llmBackend ? { llmBackend: options.llmBackend } : {}),
...(options.anthropicApiKeyEnv ? { anthropicApiKeyEnv: options.anthropicApiKeyEnv } : {}), ...(options.anthropicApiKeyEnv ? { anthropicApiKeyEnv: options.anthropicApiKeyEnv } : {}),
...(options.anthropicApiKeyFile ? { anthropicApiKeyFile: options.anthropicApiKeyFile } : {}), ...(options.anthropicApiKeyFile ? { anthropicApiKeyFile: options.anthropicApiKeyFile } : {}),
...(options.llmModel ? { llmModel: options.llmModel } : {}),
...(options.vertexProject ? { vertexProject: options.vertexProject } : {}), ...(options.vertexProject ? { vertexProject: options.vertexProject } : {}),
...(options.vertexLocation ? { vertexLocation: options.vertexLocation } : {}), ...(options.vertexLocation ? { vertexLocation: options.vertexLocation } : {}),
skipLlm: options.skipLlm === true, skipLlm: options.skipLlm === true,

View file

@ -44,7 +44,7 @@ export function registerSlCommands(program: Command, context: KtxCliCommandConte
.description('List, search, validate, or query local semantic-layer sources') .description('List, search, validate, or query local semantic-layer sources')
.usage('[options] [query...]') .usage('[options] [query...]')
.argument('[query...]', 'Search query; omit to list all sources') .argument('[query...]', 'Search query; omit to list all sources')
.option('--connection-id <id>', 'KTX connection id') .option('--connection-id <id>', 'ktx connection id')
.option('--limit <number>', 'Maximum search results (search mode only)', parsePositiveIntegerOption) .option('--limit <number>', 'Maximum search results (search mode only)', parsePositiveIntegerOption)
.addOption( .addOption(
new Option('--output <mode>', 'Output mode: pretty (default in TTY), plain (TSV), or json').choices([ new Option('--output <mode>', 'Output mode: pretty (default in TTY), plain (TSV), or json').choices([
@ -94,19 +94,28 @@ export function registerSlCommands(program: Command, context: KtxCliCommandConte
}, },
); );
sl.command('validate') sl.command('read')
.description('Validate a semantic-layer source (set --connection-id on `ktx sl`)') .description('Read a semantic-layer source YAML file')
.argument('<sourceName>', 'Semantic-layer source name')
.action(async (sourceName: string, _options, command) => {
const parentOpts = command.parent?.opts() as { connectionId?: string } | undefined;
await runSlArgs(context, {
command: 'read',
projectDir: resolveCommandProjectDir(command),
connectionId: parentOpts?.connectionId,
sourceName,
});
});
sl.command('validate')
.description('Validate a semantic-layer source')
.argument('<sourceName>', 'Semantic-layer source name') .argument('<sourceName>', 'Semantic-layer source name')
.action(async (sourceName: string, _options, command) => { .action(async (sourceName: string, _options, command) => {
const parentOpts = command.parent?.opts() as { connectionId?: string } | undefined; const parentOpts = command.parent?.opts() as { connectionId?: string } | undefined;
const connectionId = parentOpts?.connectionId;
if (connectionId === undefined) {
command.error("error: required option '--connection-id <id>' not specified");
}
await runSlArgs(context, { await runSlArgs(context, {
command: 'validate', command: 'validate',
projectDir: resolveCommandProjectDir(command), projectDir: resolveCommandProjectDir(command),
connectionId: connectionId as string, connectionId: parentOpts?.connectionId,
sourceName, sourceName,
}); });
}); });
@ -131,10 +140,14 @@ export function registerSlCommands(program: Command, context: KtxCliCommandConte
throw new Error('sl query requires at least one --measure'); throw new Error('sl query requires at least one --measure');
} }
const parentOpts = command.parent?.opts() as { connectionId?: string } | undefined; const parentOpts = command.parent?.opts() as { connectionId?: string } | undefined;
const connectionId = parentOpts?.connectionId;
if (connectionId === undefined) {
command.error("error: required option '--connection-id <id>' not specified");
}
const args = slQueryCommandSchema.parse({ const args = slQueryCommandSchema.parse({
command: 'query', command: 'query',
projectDir: resolveCommandProjectDir(command), projectDir: resolveCommandProjectDir(command),
connectionId: parentOpts?.connectionId, connectionId,
...(options.queryFile ...(options.queryFile
? { queryFile: options.queryFile } ? { queryFile: options.queryFile }
: { : {

View file

@ -26,7 +26,7 @@ export function registerSqlCommands(program: Command, context: KtxCliCommandCont
.command('sql') .command('sql')
.description('Execute parser-validated read-only SQL against a configured connection') .description('Execute parser-validated read-only SQL against a configured connection')
.argument('<sql...>', 'SQL query to execute') .argument('<sql...>', 'SQL query to execute')
.requiredOption('-c, --connection <id>', 'KTX connection id') .requiredOption('-c, --connection <id>', 'ktx connection id')
.option('--max-rows <n>', 'Maximum rows to return', parseSqlMaxRowsOption, DEFAULT_MAX_ROWS) .option('--max-rows <n>', 'Maximum rows to return', parseSqlMaxRowsOption, DEFAULT_MAX_ROWS)
.addOption( .addOption(
new Option('--output <mode>', 'Output mode: pretty (default), plain (TSV), or json').choices([ new Option('--output <mode>', 'Output mode: pretty (default), plain (TSV), or json').choices([

View file

@ -15,7 +15,7 @@ function inputMode(options: { input?: boolean }): { inputMode?: 'disabled' } {
export function registerStatusCommands(program: Command, context: KtxCliCommandContext): void { export function registerStatusCommands(program: Command, context: KtxCliCommandContext): void {
program program
.command('status') .command('status')
.description('Check current KTX setup and project readiness') .description('Check current ktx setup and project readiness')
.option('--json', 'Print JSON output', false) .option('--json', 'Print JSON output', false)
.option('-v, --verbose', 'Show every check, including passing ones', false) .option('-v, --verbose', 'Show every check, including passing ones', false)
.option('--validate', 'Only validate the ktx.yaml schema; skip readiness checks', false) .option('--validate', 'Only validate the ktx.yaml schema; skip readiness checks', false)

View file

@ -0,0 +1,28 @@
import type { KtxCliIo } from './cli-runtime.js';
import { isWritableTtyOutput } from './io/tty.js';
import { dim } from './io/symbols.js';
import { SLACK_URL } from './links.js';
type ErrorCtaVariant = 'error' | 'crash';
/** @internal */
export const SLACK_HELP_FOOTER = `Community & support: ${SLACK_URL}`;
/** @internal */
export const SLACK_SETUP_NOTE = {
title: 'Community',
body: `Questions or feedback? Join the ktx Slack: ${SLACK_URL}`,
} as const;
export function writeErrorCommunityHint(io: KtxCliIo, variant: ErrorCtaVariant): void {
if (!isWritableTtyOutput(io.stderr)) {
return;
}
const line =
variant === 'crash'
? `This may be a bug - report it or ask in the ktx community: ${SLACK_URL}`
: `Stuck? The ktx community can help: ${SLACK_URL}`;
io.stderr.write(`${dim(line)}\n`);
}

View file

@ -0,0 +1,172 @@
import type { CommandUnknownOpts, Option } from '@commander-js/extra-typings';
/**
* Dynamic completion candidates that depend on project state (semantic-layer
* source names, wiki page keys, connection ids). Injected so the engine stays
* pure and unit-testable without touching the filesystem.
*/
export interface CompletionProviders {
/** Candidate operands for a positional argument of the active command path. */
positionalCandidates(commandPath: string[], typedTokens: string[]): Promise<string[]>;
/** Candidate values for an option that has no static `choices` (e.g. `--connection-id`). */
optionValueCandidates(commandPath: string[], optionFlag: string, typedTokens: string[]): Promise<string[]>;
}
interface ResolvedCommand {
command: CommandUnknownOpts;
/** Subcommand names from the root down to the active command (root name excluded). */
commandPath: string[];
}
function isHiddenCommand(command: CommandUnknownOpts): boolean {
// Completion mirrors `ktx --help`: commands registered with `{ hidden: true }`
// (the `__complete` helper and `mcp serve-internal`) are internal and must not
// surface. Commander exposes this only through the private `_hidden` field its
// own help renderer reads, so a name heuristic like a `__` prefix is not enough.
return (command as { _hidden?: boolean })._hidden === true;
}
function resolveCommand(program: CommandUnknownOpts, typedTokens: string[]): ResolvedCommand {
let command: CommandUnknownOpts = program;
const commandPath: string[] = [];
for (let index = 0; index < typedTokens.length; index += 1) {
const token = typedTokens[index];
if (token.startsWith('-')) {
// A value-taking option in the `--flag value` form consumes the next token
// as its value, so skip that value before matching subcommands. Otherwise a
// connection id like `query` would be resolved as the `sl query` subcommand
// instead of being treated as the `--connection-id` value. The `--flag=value`
// form carries its own value and consumes nothing extra.
if (!token.includes('=')) {
const option = findOption(command, token);
if (option && !option.isBoolean()) {
index += 1;
}
}
continue;
}
const sub = command.commands.find((candidate) => candidate.name() === token || candidate.aliases().includes(token));
if (sub) {
command = sub;
commandPath.push(sub.name());
}
}
return { command, commandPath };
}
function collectOptions(command: CommandUnknownOpts): Option[] {
const options: Option[] = [];
let current: CommandUnknownOpts | null = command;
while (current) {
options.push(...current.options);
current = current.parent;
}
return options;
}
function findOption(command: CommandUnknownOpts, flag: string): Option | undefined {
return collectOptions(command).find((option) => option.long === flag || option.short === flag);
}
function isRepeatableOption(option: Option): boolean {
// Variadic options, and options backed by a collector with an array default
// (e.g. `--measure`/`--dimension`), may be supplied more than once.
return option.variadic || Array.isArray(option.defaultValue);
}
function flagCandidates(command: CommandUnknownOpts, typedTokens: string[]): string[] {
const present = new Set(typedTokens.filter((token) => token.startsWith('-')));
const candidates: string[] = [];
for (const option of collectOptions(command)) {
if (option.hidden || !option.long) {
continue;
}
if (present.has(option.long) && !isRepeatableOption(option)) {
continue;
}
candidates.push(option.long);
}
return candidates;
}
async function optionValueCandidates(
resolved: ResolvedCommand,
option: Option,
typedTokens: string[],
providers: CompletionProviders,
): Promise<string[]> {
if (option.argChoices && option.argChoices.length > 0) {
return option.argChoices;
}
return providers.optionValueCandidates(resolved.commandPath, option.long ?? option.name(), typedTokens);
}
function dedupeSortFilter(candidates: string[], partial: string): string[] {
const seen = new Set<string>();
const matches: string[] = [];
for (const candidate of candidates) {
if (!candidate.startsWith(partial) || seen.has(candidate)) {
continue;
}
seen.add(candidate);
matches.push(candidate);
}
return matches.sort();
}
/**
* Compute completion candidates for the partial last element of `words`
* (everything the shell has on the line after `ktx`). The active command and
* its flags are derived by walking the live Commander tree, so completion never
* drifts from the real command structure.
*/
export async function computeCompletions(
program: CommandUnknownOpts,
words: string[],
providers: CompletionProviders,
): Promise<string[]> {
const partial = words.length > 0 ? (words[words.length - 1] ?? '') : '';
const typedTokens = words.slice(0, -1);
const resolved = resolveCommand(program, typedTokens);
// (a) Option value via the `--opt=value` form.
const equalsMatch = /^(--[^=]+)=(.*)$/.exec(partial);
if (equalsMatch) {
const [, flag, valuePartial] = equalsMatch;
const option = findOption(resolved.command, flag);
if (!option || option.isBoolean()) {
return [];
}
const values = await optionValueCandidates(resolved, option, typedTokens, providers);
return dedupeSortFilter(
values.map((value) => `${flag}=${value}`),
`${flag}=${valuePartial}`,
);
}
// (b) Option value via the `--opt value` form (previous token is a value-taking option).
const previous = typedTokens[typedTokens.length - 1];
if (previous && previous.startsWith('-') && !partial.startsWith('-')) {
const option = findOption(resolved.command, previous);
if (option && !option.isBoolean()) {
return dedupeSortFilter(await optionValueCandidates(resolved, option, typedTokens, providers), partial);
}
}
// (c) Flag completion.
if (partial.startsWith('-')) {
return dedupeSortFilter(flagCandidates(resolved.command, typedTokens), partial);
}
// (d) Positional: subcommand names union static argument choices union dynamic operand candidates.
const candidates: string[] = resolved.command.commands
.filter((sub) => !isHiddenCommand(sub))
.map((sub) => sub.name());
for (const argument of resolved.command.registeredArguments) {
if (argument.argChoices) {
candidates.push(...argument.argChoices);
}
}
candidates.push(...(await providers.positionalCandidates(resolved.commandPath, typedTokens)));
return dedupeSortFilter(candidates, partial);
}

View file

@ -0,0 +1,39 @@
// Static shell completion scripts emitted by `ktx completion <shell>`.
//
// Both scripts gather the words on the current command line (excluding the
// leading `ktx`), append the partial word under the cursor, and delegate to the
// hidden `ktx __complete` command, which prints newline-separated candidates.
// All command/flag/entity knowledge lives in `ktx __complete` so these scripts
// never have to encode the command tree.
//
// Lines are single-quoted JS strings so the shell `${...}` expansions are
// emitted verbatim (a template literal would try to interpolate them).
const ZSH_SCRIPT = [
'#compdef ktx',
'_ktx() {',
' local -a candidates',
' local out',
' out="$(ktx __complete -- "${words[@]:1:$((CURRENT-1))}" 2>/dev/null)" || return 0',
' candidates=("${(@f)out}")',
' compadd -- $candidates',
'}',
'compdef _ktx ktx',
'',
].join('\n');
const BASH_SCRIPT = [
'_ktx() {',
' local cur out',
' cur="${COMP_WORDS[COMP_CWORD]}"',
' out="$(ktx __complete -- "${COMP_WORDS[@]:1:COMP_CWORD}" 2>/dev/null)" || { COMPREPLY=(); return 0; }',
" local IFS=$'\\n'",
' COMPREPLY=($(compgen -W "${out}" -- "$cur"))',
'}',
'complete -F _ktx ktx',
'',
].join('\n');
export function completionScript(shell: 'zsh' | 'bash'): string {
return shell === 'zsh' ? ZSH_SCRIPT : BASH_SCRIPT;
}

View file

@ -0,0 +1,103 @@
import { existsSync } from 'node:fs';
import { join } from 'node:path';
import type { KtxLocalProject } from '../context/project/project.js';
import { resolveKtxProjectDir } from '../project-resolver.js';
import type { CompletionProviders } from './complete-engine.js';
/** Extract an option value from already-typed tokens (`--flag value` or `--flag=value`). */
function extractOptionValue(tokens: string[], flag: string): string | undefined {
const prefix = `${flag}=`;
for (let index = 0; index < tokens.length; index += 1) {
const token = tokens[index];
if (token === flag) {
const next = tokens[index + 1];
if (next !== undefined && !next.startsWith('-')) {
return next;
}
} else if (token.startsWith(prefix)) {
return token.slice(prefix.length);
}
}
return undefined;
}
/**
* Resolve and load the project the user is completing against. Honors a
* `--project-dir` typed on the line, then `KTX_PROJECT_DIR`, then the nearest
* `ktx.yaml`. Returns null (no completions) when there is no project, without
* creating any files.
*/
async function loadCompletionProject(typedTokens: string[]): Promise<KtxLocalProject | null> {
const explicitProjectDir = extractOptionValue(typedTokens, '--project-dir');
const projectDir = resolveKtxProjectDir(explicitProjectDir !== undefined ? { explicitProjectDir } : {});
if (!existsSync(join(projectDir, 'ktx.yaml'))) {
return null;
}
const { loadKtxProject } = await import('../context/project/project.js');
return loadKtxProject({ projectDir });
}
async function sourceNames(typedTokens: string[]): Promise<string[]> {
const project = await loadCompletionProject(typedTokens);
if (!project) {
return [];
}
const connectionId = extractOptionValue(typedTokens, '--connection-id');
const { listLocalSlSources } = await import('../context/sl/local-sl.js');
const summaries = await listLocalSlSources(project, connectionId !== undefined ? { connectionId } : {});
return [...new Set(summaries.map((summary) => summary.name))];
}
async function wikiPageKeys(typedTokens: string[]): Promise<string[]> {
const project = await loadCompletionProject(typedTokens);
if (!project) {
return [];
}
const userId = extractOptionValue(typedTokens, '--user-id');
const { listLocalKnowledgePageKeys } = await import('../context/wiki/local-knowledge.js');
return listLocalKnowledgePageKeys(project, userId !== undefined ? { userId } : {});
}
async function connectionIds(typedTokens: string[]): Promise<string[]> {
const project = await loadCompletionProject(typedTokens);
if (!project) {
return [];
}
return Object.keys(project.config.connections).sort();
}
/**
* Project-backed completion providers. Every entry swallows its own errors so a
* failed lookup never breaks the shell completion degrades to commands/flags.
*/
export function createProjectCompletionProviders(): CompletionProviders {
return {
async positionalCandidates(commandPath, typedTokens) {
try {
const key = commandPath.join(' ');
if (key === 'sl read' || key === 'sl validate') {
return await sourceNames(typedTokens);
}
if (key === 'wiki read') {
return await wikiPageKeys(typedTokens);
}
if (key === 'connection test' || key === 'ingest') {
return await connectionIds(typedTokens);
}
return [];
} catch {
return [];
}
},
async optionValueCandidates(_commandPath, optionFlag, typedTokens) {
try {
if (optionFlag === '--connection-id' || optionFlag === '--connection') {
return await connectionIds(typedTokens);
}
return [];
} catch {
return [];
}
},
};
}

View file

@ -0,0 +1,21 @@
import type { KtxProjectConnectionConfig } from './context/project/config.js';
const KTX_DATABASE_DRIVER_IDS = new Set([
'sqlite',
'postgres',
'mysql',
'clickhouse',
'sqlserver',
'bigquery',
'snowflake',
]);
export function normalizeConnectionDriver(connection: KtxProjectConnectionConfig): string {
return String(connection.driver ?? '')
.trim()
.toLowerCase();
}
export function isDatabaseDriver(driver: string): boolean {
return KTX_DATABASE_DRIVER_IDS.has(driver.trim().toLowerCase());
}

View file

@ -0,0 +1,132 @@
import type { KtxCliIo } from './cli-runtime.js';
import type { KtxSetupPromptOption } from './setup-prompts.js';
export type RecoveryOutcome = 'ready' | 'skip' | 'back' | 'failed';
/** @internal */
export interface RecoveryAction {
value: string;
label: string;
run: () => Promise<void>;
}
export type ConfigureResult = 'configured' | 'back' | 'cancelled';
export type ValidateResult =
| { status: 'ok' }
| { status: 'back' }
| { status: 'failed'; extraActions?: RecoveryAction[] };
export interface ConnectionRecoveryInput {
label: string;
interactive: boolean;
allowSkip: boolean;
io: KtxCliIo;
prompts: {
select(options: { message: string; options: KtxSetupPromptOption[] }): Promise<string>;
};
snapshot: () => Promise<() => Promise<void>>;
configure: () => Promise<ConfigureResult>;
validate: () => Promise<ValidateResult>;
}
async function runRollbackOnce(input: {
rollback: () => Promise<void>;
state: { rolledBack: boolean };
}): Promise<void> {
if (input.state.rolledBack) {
return;
}
input.state.rolledBack = true;
await input.rollback();
}
function recoveryOptions(input: {
allowSkip: boolean;
extraActions?: RecoveryAction[];
}): KtxSetupPromptOption[] {
return [
{ value: 'retry', label: 'Retry connection test' },
{ value: 're-enter', label: 'Re-enter connection details' },
...(input.extraActions ?? []).map((action) => ({
value: action.value,
label: action.label,
})),
...(input.allowSkip ? [{ value: 'skip', label: 'Skip this connection' }] : []),
{ value: 'back', label: 'Back' },
];
}
export async function runConnectionSetupWithRecovery(
input: ConnectionRecoveryInput,
): Promise<RecoveryOutcome> {
const rollback = await input.snapshot();
const rollbackState = { rolledBack: false };
const firstConfig = await input.configure();
if (firstConfig === 'back') {
await runRollbackOnce({ rollback, state: rollbackState });
return 'back';
}
if (firstConfig === 'cancelled') {
await runRollbackOnce({ rollback, state: rollbackState });
return 'failed';
}
let validation = await input.validate();
while (validation.status !== 'ok') {
if (validation.status === 'back') {
await runRollbackOnce({ rollback, state: rollbackState });
return 'back';
}
if (!input.interactive) {
return 'failed';
}
const action = await input.prompts.select({
message: `Connection setup failed for ${input.label}`,
options: recoveryOptions({
allowSkip: input.allowSkip,
extraActions: validation.extraActions,
}),
});
if (action === 'back') {
await runRollbackOnce({ rollback, state: rollbackState });
return 'back';
}
if (action === 'skip' && input.allowSkip) {
await runRollbackOnce({ rollback, state: rollbackState });
return 'skip';
}
if (action === 're-enter') {
const nextConfig = await input.configure();
if (nextConfig === 'back') {
await runRollbackOnce({ rollback, state: rollbackState });
return 'back';
}
if (nextConfig === 'cancelled') {
await runRollbackOnce({ rollback, state: rollbackState });
return 'failed';
}
validation = await input.validate();
continue;
}
if (action === 'retry') {
validation = await input.validate();
continue;
}
const extraAction = validation.extraActions?.find((candidate) => candidate.value === action);
if (extraAction) {
await extraAction.run();
validation = await input.validate();
continue;
}
validation = await input.validate();
}
return 'ready';
}

View file

@ -16,8 +16,9 @@ import { bold, dim, green, red, SYMBOLS } from './io/symbols.js';
import { createKtxCliScanConnector } from './local-scan-connectors.js'; import { createKtxCliScanConnector } from './local-scan-connectors.js';
import { profileMark } from './startup-profile.js'; import { profileMark } from './startup-profile.js';
import { isDemoConnection } from './telemetry/demo-detect.js'; import { isDemoConnection } from './telemetry/demo-detect.js';
import { emitTelemetryEvent } from './telemetry/index.js'; import { emitTelemetryEvent, reportException } from './telemetry/index.js';
import { scrubErrorClass } from './telemetry/scrubber.js'; import { collectTelemetryRedactionSecrets } from './telemetry/redaction-secrets.js';
import { formatErrorDetail, scrubErrorClass } from './telemetry/scrubber.js';
profileMark('module:connection'); profileMark('module:connection');
@ -74,6 +75,12 @@ async function testNativeConnection(
} }
const result = await connector.testConnection(); const result = await connector.testConnection();
if (!result.success) { if (!result.success) {
// Re-throw the driver's original error so connection_test telemetry records
// its real class (e.g. ConnectionError) and code (e.g. ELOGIN) instead of
// collapsing every native failure to a generic Error with no code.
if (result.cause instanceof Error) {
throw result.cause;
}
throw new Error(result.error ?? 'connection test failed'); throw new Error(result.error ?? 'connection test failed');
} }
return { driver: connector.driver }; return { driver: connector.driver };
@ -127,7 +134,7 @@ async function createDefaultLookerClient(
connectionId: string, connectionId: string,
): Promise<LookerTestPort> { ): Promise<LookerTestPort> {
const factory = new DefaultLookerConnectionClientFactory(createLocalLookerCredentialResolver(project)); const factory = new DefaultLookerConnectionClientFactory(createLocalLookerCredentialResolver(project));
return (await factory.createClient(connectionId)) as unknown as LookerTestPort; return factory.createLookerClient(connectionId);
} }
async function testLookerConnection( async function testLookerConnection(
@ -304,6 +311,7 @@ async function emitConnectionTest(input: {
io: KtxCliIo; io: KtxCliIo;
}): Promise<void> { }): Promise<void> {
const errorClass = input.error ? scrubErrorClass(input.error) : undefined; const errorClass = input.error ? scrubErrorClass(input.error) : undefined;
const errorDetail = input.error ? formatErrorDetail(input.error) : undefined;
await emitTelemetryEvent({ await emitTelemetryEvent({
name: 'connection_test', name: 'connection_test',
projectDir: input.project.projectDir, projectDir: input.project.projectDir,
@ -314,8 +322,24 @@ async function emitConnectionTest(input: {
outcome: input.outcome, outcome: input.outcome,
durationMs: input.durationMs, durationMs: input.durationMs,
...(errorClass ? { errorClass } : {}), ...(errorClass ? { errorClass } : {}),
...(errorDetail ? { errorDetail } : {}),
}, },
}); });
if (input.error) {
await reportException({
error: input.error,
context: { source: 'connection test', handled: true, fatal: false },
projectDir: input.project.projectDir,
io: input.io,
redactionSecrets: await collectTelemetryRedactionSecrets({
project: input.project,
connectionId: input.connectionId,
includeLlm: false,
includeEmbeddings: false,
env: process.env,
}),
});
}
} }
function visualWidth(text: string): number { function visualWidth(text: string): number {

View file

@ -5,7 +5,9 @@ import { assertReadOnlySql, limitSqlForExecution } from '../../context/connectio
import { tryConstraintQuery } from '../../context/scan/constraint-discovery.js'; import { tryConstraintQuery } from '../../context/scan/constraint-discovery.js';
import { scopedTableNames } from '../../context/scan/table-ref.js'; import { scopedTableNames } from '../../context/scan/table-ref.js';
import { import {
connectorTestFailure,
createKtxConnectorCapabilities, createKtxConnectorCapabilities,
type KtxConnectorTestResult,
type KtxColumnSampleInput, type KtxColumnSampleInput,
type KtxColumnSampleResult, type KtxColumnSampleResult,
type KtxColumnStatsInput, type KtxColumnStatsInput,
@ -320,7 +322,7 @@ export class KtxBigQueryScanConnector implements KtxScanConnector {
this.id = `bigquery:${options.connectionId}`; this.id = `bigquery:${options.connectionId}`;
} }
async testConnection(): Promise<{ success: boolean; error?: string }> { async testConnection(): Promise<KtxConnectorTestResult> {
try { try {
const client = this.getClient(); const client = this.getClient();
await client.getDatasets({ maxResults: 1 }); await client.getDatasets({ maxResults: 1 });
@ -329,7 +331,7 @@ export class KtxBigQueryScanConnector implements KtxScanConnector {
} }
return { success: true }; return { success: true };
} catch (error) { } catch (error) {
return { success: false, error: error instanceof Error ? error.message : String(error) }; return connectorTestFailure(error);
} }
} }

View file

@ -1,7 +1,7 @@
import { createClient } from '@clickhouse/client'; import { createClient } from '@clickhouse/client';
import { getDialectForDriver } from '../../context/connections/dialects.js'; import { getDialectForDriver } from '../../context/connections/dialects.js';
import { assertReadOnlySql, limitSqlForExecution } from '../../context/connections/read-only-sql.js'; import { assertReadOnlySql, limitSqlForExecution } from '../../context/connections/read-only-sql.js';
import { createKtxConnectorCapabilities, type KtxColumnSampleInput, type KtxColumnSampleResult, type KtxColumnStatsInput, type KtxColumnStatsResult, type KtxQueryResult, type KtxReadOnlyQueryInput, type KtxScanConnector, type KtxScanContext, type KtxScanInput, type KtxSchemaColumn, type KtxSchemaSnapshot, type KtxSchemaTable, type KtxTableRef, type KtxTableSampleInput, type KtxTableListEntry, type KtxTableSampleResult } from '../../context/scan/types.js'; import { connectorTestFailure, createKtxConnectorCapabilities, type KtxConnectorTestResult, type KtxColumnSampleInput, type KtxColumnSampleResult, type KtxColumnStatsInput, type KtxColumnStatsResult, type KtxQueryResult, type KtxReadOnlyQueryInput, type KtxScanConnector, type KtxScanContext, type KtxScanInput, type KtxSchemaColumn, type KtxSchemaSnapshot, type KtxSchemaTable, type KtxTableRef, type KtxTableSampleInput, type KtxTableListEntry, type KtxTableSampleResult } from '../../context/scan/types.js';
import { scopedTableNames } from '../../context/scan/table-ref.js'; import { scopedTableNames } from '../../context/scan/table-ref.js';
import { readFileSync } from 'node:fs'; import { readFileSync } from 'node:fs';
import { Agent as HttpsAgent } from 'node:https'; import { Agent as HttpsAgent } from 'node:https';
@ -317,12 +317,12 @@ export class KtxClickHouseScanConnector implements KtxScanConnector {
this.id = `clickhouse:${options.connectionId}`; this.id = `clickhouse:${options.connectionId}`;
} }
async testConnection(): Promise<{ success: boolean; error?: string }> { async testConnection(): Promise<KtxConnectorTestResult> {
try { try {
await this.query('SELECT 1'); await this.query('SELECT 1');
return { success: true }; return { success: true };
} catch (error) { } catch (error) {
return { success: false, error: error instanceof Error ? error.message : String(error) }; return connectorTestFailure(error);
} }
} }
@ -645,7 +645,7 @@ export class KtxClickHouseScanConnector implements KtxScanConnector {
private assertConnection(connectionId: string): void { private assertConnection(connectionId: string): void {
if (connectionId !== this.connectionId) { if (connectionId !== this.connectionId) {
throw new Error(`KTX ClickHouse connector ${this.id} cannot serve connection ${connectionId}`); throw new Error(`ktx ClickHouse connector ${this.id} cannot serve connection ${connectionId}`);
} }
} }
} }

View file

@ -11,7 +11,9 @@ import {
} from '../../context/scan/constraint-discovery.js'; } from '../../context/scan/constraint-discovery.js';
import { scopedTableNames } from '../../context/scan/table-ref.js'; import { scopedTableNames } from '../../context/scan/table-ref.js';
import { import {
connectorTestFailure,
createKtxConnectorCapabilities, createKtxConnectorCapabilities,
type KtxConnectorTestResult,
type KtxColumnSampleInput, type KtxColumnSampleInput,
type KtxColumnSampleResult, type KtxColumnSampleResult,
type KtxColumnStatsInput, type KtxColumnStatsInput,
@ -157,6 +159,15 @@ interface MysqlDistinctValueRow extends RowDataPacket {
val: unknown; val: unknown;
} }
interface MysqlStatsRow extends RowDataPacket {
column_name: string;
estimated_cardinality: number | null;
}
export interface KtxMysqlColumnStatisticsResult {
cardinalityByColumn: Map<string, number>;
}
class DefaultMysqlPoolFactory implements KtxMysqlPoolFactory { class DefaultMysqlPoolFactory implements KtxMysqlPoolFactory {
createPool(config: KtxMysqlPoolConfig): KtxMysqlPool { createPool(config: KtxMysqlPoolConfig): KtxMysqlPool {
return mysql.createPool(config) as Pool; return mysql.createPool(config) as Pool;
@ -382,7 +393,7 @@ export class KtxMysqlScanConnector implements KtxScanConnector {
readonly capabilities = createKtxConnectorCapabilities({ readonly capabilities = createKtxConnectorCapabilities({
tableSampling: true, tableSampling: true,
columnSampling: true, columnSampling: true,
columnStats: false, columnStats: true,
readOnlySql: true, readOnlySql: true,
nestedAnalysis: true, nestedAnalysis: true,
formalForeignKeys: true, formalForeignKeys: true,
@ -413,12 +424,12 @@ export class KtxMysqlScanConnector implements KtxScanConnector {
this.id = `mysql:${options.connectionId}`; this.id = `mysql:${options.connectionId}`;
} }
async testConnection(): Promise<{ success: boolean; error?: string }> { async testConnection(): Promise<KtxConnectorTestResult> {
try { try {
await this.query('SELECT 1'); await this.query('SELECT 1');
return { success: true }; return { success: true };
} catch (error) { } catch (error) {
return { success: false, error: error instanceof Error ? error.message : String(error) }; return connectorTestFailure(error);
} }
} }
@ -560,9 +571,30 @@ export class KtxMysqlScanConnector implements KtxScanConnector {
return { values, nullCount: null, distinctCount: null }; return { values, nullCount: null, distinctCount: null };
} }
async columnStats(_input: KtxColumnStatsInput, _ctx: KtxScanContext): Promise<KtxColumnStatsResult | null> { async columnStats(input: KtxColumnStatsInput, _ctx: KtxScanContext): Promise<KtxColumnStatsResult | null> {
const stats = await this.getColumnStatistics(input.table);
const value = stats?.cardinalityByColumn.get(input.column);
return value === undefined
? null
: { min: null, max: null, average: null, nullCount: null, distinctCount: value };
}
async getColumnStatistics(table: KtxTableRef): Promise<KtxMysqlColumnStatisticsResult | null> {
const schema = table.db ?? this.poolConfig.database;
const sql = this.dialect.generateColumnStatisticsQuery(schema, table.name);
if (!sql) {
return null; return null;
} }
const rows = await this.queryRaw<MysqlStatsRow>(sql);
const cardinalityByColumn = new Map<string, number>();
for (const row of rows) {
const cardinality = Number(row.estimated_cardinality);
if (Number.isFinite(cardinality) && cardinality >= 0) {
cardinalityByColumn.set(row.column_name, cardinality);
}
}
return cardinalityByColumn.size > 0 ? { cardinalityByColumn } : null;
}
async executeReadOnly(input: KtxMysqlReadOnlyQueryInput, _ctx: KtxScanContext): Promise<KtxQueryResult> { async executeReadOnly(input: KtxMysqlReadOnlyQueryInput, _ctx: KtxScanContext): Promise<KtxQueryResult> {
this.assertConnection(input.connectionId); this.assertConnection(input.connectionId);
@ -762,7 +794,7 @@ export class KtxMysqlScanConnector implements KtxScanConnector {
private assertConnection(connectionId: string): void { private assertConnection(connectionId: string): void {
if (connectionId !== this.connectionId) { if (connectionId !== this.connectionId) {
throw new Error(`KTX MySQL connector ${this.id} cannot serve connection ${connectionId}`); throw new Error(`ktx MySQL connector ${this.id} cannot serve connection ${connectionId}`);
} }
} }
} }

Some files were not shown because too many files have changed in this diff Show more