Compare commits

..

No commits in common. "main" and "v0.7.0" have entirely different histories.
main ... v0.7.0

787 changed files with 13346 additions and 63764 deletions

View file

@ -217,7 +217,7 @@ jobs:
flags: typescript
name: typescript
disable_search: true
fail_ci_if_error: false
fail_ci_if_error: true
- name: Warn when Codecov token is missing for TypeScript
if: env.CODECOV_TOKEN_CONFIGURED != 'true'
@ -236,7 +236,7 @@ jobs:
flags: python
name: python
disable_search: true
fail_ci_if_error: false
fail_ci_if_error: true
- name: Warn when Codecov token is missing for Python
if: env.CODECOV_TOKEN_CONFIGURED != 'true'

View file

@ -22,7 +22,7 @@ jobs:
github.event.issue.author_association != 'COLLABORATOR'
steps:
- name: Apply needs-triage label
uses: actions/github-script@v9
uses: actions/github-script@v7
with:
script: |
await github.rest.issues.addLabels({

View file

@ -14,18 +14,6 @@ repos:
- id: check-case-conflict
- id: mixed-line-ending
- repo: https://github.com/tombi-toml/tombi-pre-commit
rev: v1.1.0
hooks:
- id: tombi-format
args: ["--offline"]
# uv.lock is generated and owned by uv, which writes its own canonical
# TOML layout. tombi reformats that layout differently, so once uv
# regenerates the lock (e.g. after a dependency or version change)
# tombi rewrites it and the hook fails on the modified file. Keep uv
# authoritative for its lockfile; tombi still formats hand-edited TOML.
exclude: ^uv\.lock$
- repo: https://github.com/asottile/pyupgrade
rev: v3.21.2
hooks:

162
AGENTS.md
View file

@ -64,25 +64,6 @@ When rules conflict, follow this order:
4. Code quality: types, readable boundaries, focused modules
5. Performance where it matters
## Opinionated Product Defaults
- **MUST**: Prefer one canonical behavior over configurable alternatives. A new
flag, config field, environment variable, mode, strategy option, adapter hook,
or fallback path is a product feature and must be justified by an explicit
user request or a real correctness requirement.
- **MUST NOT**: Add speculative flexibility for imagined users, migrations,
review preferences, local workflows, or "just in case" scenarios. If the
requested behavior can work with one solid default, implement that default.
- **MUST NOT**: Add boolean switches that create two runtime paths unless both
paths are essential and the user explicitly asked for the choice. Boolean
policy knobs are especially suspect because they double the state space and
test surface.
- **MUST**: When a design seems to need a new option, first try to remove the
need by choosing the stronger default, tightening the invariant, or failing
clearly. Ask the user before adding the option if it still seems necessary.
- **MUST**: Delete obsolete branches, tests, docs, and config when removing a
behavior. Do not preserve dormant compatibility paths.
## Repository Shape
**ktx** is a pnpm + uv workspace.
@ -178,120 +159,6 @@ and naming asymmetries are bugs in waiting — see
[`docs/code-design.md`](docs/code-design.md). Treat the `MUST` / `MUST NOT`
rules there with the same weight as the ones in this file.
## Design Reasoning Defaults
When proposing a design, an approach, or any non-trivial change, apply these
defaults and run the self-check before presenting it. They encode the
corrections users most often have to make; reaching these conclusions
autonomously — without being asked the leading question — is the bar.
- **MUST**: Optimize for the best outcome, not for an unstated constraint. Do not
silently adopt "smallest change", "least effort", "cheapest", or "least user
intervention" as the goal unless the user said so. Default to the most correct,
durable solution, and present cost / effort / scope as information for the user
to weigh — not as a ceiling you impose on their behalf.
- **MUST**: Separate one-time cost from recurring cost before discarding an
option. A fixed cost paid once (a setup-time computation, an extra LLM call
during setup, a contract change) to make every later run cheaper or more
correct is usually worth it. Do not reject it with recurring-cost reasoning;
quantify both sides. (Example smell: "don't add an LLM call to a cost-cutting
feature" — wrong when the call is one-time and the savings recur.)
- **MUST**: Treat a user's example as a representative of a class, not as the
spec. Design for the general population the example stands for, then stress-test
against deliberately different instances — another warehouse, dialect, stack
layout, or input shape — before committing. If a design only works because of an
incidental property of the example (e.g. "the noise happened to be in a separate
schema *on this demo*"), it is curve-fitting; generalize it or state the
assumption explicitly.
- **MUST**: Prefer deriving from the system's own state over enumerating cases.
Favor an allowlist computed from declared/observed state (config, scanned
catalog, query log, the user's own inputs) over a denylist of known-bad
specifics (particular tables, schemas, tools, or vendors). A hardcoded or
hand-maintained list of external specifics is a smell: it rots and fails on the
next stack. The only acceptable static patterns are genuinely universal
invariants (e.g. DB-engine system catalogs) and ktx's own self-emitted
signatures.
- **MUST**: Give each capability one implementation and route every caller
through it. When some behavior — running a query, resolving a credential or
config reference, authenticating, selecting a dialect, loading config —
already has a working implementation that some call sites use, make new or
divergent call sites depend on that path instead of standing up a second one.
Parallel implementations of one capability drift apart silently: a fix, a
newly supported input, or an added case lands on one path and not the other,
so one entry point (a CLI command, an MCP tool, an ingest stage) succeeds
while another fails on the same input. When two paths already do the same
job, collapse onto the shared one and delete the duplicate instead of
keeping both. When fixing a defect that lives on one path, fix the shared
implementation; do not patch the symptom on a forked branch, which preserves
the divergence you set out to remove.
- **SHOULD**: Before inventing an abstraction or hand-rolling structural logic,
search for what already exists and reuse it — the codebase's canonical
representation (a structured ref/key type) instead of a parallel string scheme,
and a mandated/available tool (e.g. `sqlglot` for SQL structure; see
[SQL and Structured Parsing](#sql-and-structured-parsing)) instead of
hand-parsing. Normalize ambiguous input to the canonical form at the boundary;
do not carry the ambiguity downstream. This is the single-source-of-truth / DRY
item from the Priority Hierarchy applied at design time.
Before presenting a design, answer these explicitly:
1. Am I optimizing for a goal the user actually stated, or one I assumed?
2. Does this generalize beyond the example in front of me? Name a real case where
it would break.
3. Am I enumerating known-bad cases when I could derive scope from the system's
own declared/observed state?
4. Is there an existing canonical representation or mandated tool I should reuse
instead of building or parsing my own?
5. Am I discarding the better option on a weak or misapplied constraint
(one-time vs recurring cost, "more surface area", "more work now")?
6. Does another entry point already perform this operation through a shared
implementation? If so, am I routing through that path instead of forking a
parallel one — and if I'm fixing a bug, am I fixing the shared layer rather
than one branch?
7. Am I adding a user-visible option or alternate runtime path that the user did
not ask for? If yes, can one opinionated default solve the problem instead?
8. Does this option multiply behavior by caller path, config value, or local
state? If yes, remove it unless it is explicitly required.
A user question that nudges toward an alternative ("would X help?", "should I
always do Y?", "will you hardcode Z?") is a signal that a better option exists.
Investigate the implied direction and reason it through *before* defending the
original proposal — and prefer to have asked yourself the question first.
Example: If generated context changes should be saved, choose one save policy
and route ingest, setup, memory, indexing, and docs through it. Do not add an
`auto_commit`-style switch unless the user explicitly asks for staged-only runs
and accepts the extra runtime path.
## Code Comments and Docstrings
Code must be self-explanatory. Clear names, types, and signatures do the
documenting; a comment or docstring exists only to state what the code cannot
show. Everything else belongs in the PR description or nowhere.
- **MUST**: Keep each comment to 1-3 lines stating only what the code cannot
show: a cross-file invariant ("error-severity issues never reach here — the
doctor exits on them first"), a required ordering ("ktx.yaml is written
before git init, so a crash cannot leave a bare `.git`"), or a library quirk
("zod reports unknown record keys as `invalid_key`").
- **MUST**: Hold docstrings (Python `"""..."""`, JSDoc/TSDoc) to the same bar.
A docstring states a function's purpose or contract in 1-3 lines; when a real
quirk or invariant motivates the code, note it once and briefly. Let
self-explanatory code carry the rest — a well-named, well-typed function
often needs no docstring at all.
- **MUST**: State each invariant once, at the public entry point. Do not repeat
the same guarantee across a module docstring, a helper, its wrapper, and the
call site.
- **MUST NOT**: Write multi-paragraph docstrings or prose comment blocks —
design rationale, alternatives considered, change narration ("is now written
before…"), caller enumerations ("shared by X, Y, and Z"), worked examples
that restate the code, or the same explanation repeated in a module docstring
and the function it describes. That is the author addressing the reviewer; it
belongs in the PR description and rots once merged.
- **MAY**: Open a regression test with a 1-3 line comment stating the scenario
it guards when the test name cannot carry it. Omit design history and
references to removed designs.
## TypeScript Standards
- Use Node 22+ and pnpm workspace commands.
@ -411,8 +278,7 @@ use `PascalCase` without the suffix.
## Telemetry
**ktx** ships PostHog usage telemetry. Catalog telemetry events use strict
schemas. When adding commands or events:
**ktx** ships anonymous PostHog telemetry. When adding commands or events:
- **MUST NOT**: Add fields that carry user data — file paths, hostnames,
environment values, SQL text, schema/table/column names, error messages,
@ -429,24 +295,6 @@ schemas. When adding commands or events:
of collected data changes. Adding another event with no new field types
needs no docs change.
### Error reports
**ktx** also sends PostHog Error Tracking `$exception` events when telemetry is
enabled. This channel is separate from the strict catalog event schema and is
used only for exception diagnostics.
`$exception` events may include stack frames, error class names, raw error
messages, cause chains, `source`, `handled`, `fatal`, runtime version fields,
OS/runtime fields, and the hashed `projectId` when known. Stack frames may
include local file paths and the local username when those appear in paths.
`$exception` events must never intentionally include secrets, credentials,
database URLs, auth headers, raw argv, raw environment values, SQL text,
schema/table/column names as explicit properties, customer row data, user prompt
text, or raw MCP arguments. Reporters must redact call-site-provided secret
snapshots and common static credential patterns before the SDK serializes the
exception.
## Documentation and Specs
- Keep public documentation in `README.md`, package READMEs, example READMEs,
@ -502,9 +350,8 @@ error messages — including the disambiguation rule for the overloaded word
`source` (semantic / primary / context / source of truth) — see
[`docs/terminology.md`](docs/terminology.md). Follow that file when choosing
between near-synonyms (e.g. `connector` vs `adapter`, `data agent` vs
`database agent`, `context-source ingest` vs `source ingest`). Product-name
rules in this section take precedence over anything in that file when they
conflict.
`database agent`, `fast ingest` vs `schema ingest`). Product-name rules in
this section take precedence over anything in that file when they conflict.
### Updating `docs-site/` After Code Changes
@ -533,8 +380,7 @@ rather than silently skipping it.
- **MUST**: Disable monospace ligatures on every surface that uses the
`var(--font-mono)` family (Geist Mono). Geist Mono fuses `--` into an
em-dash glyph that visually eats the adjacent space, so prompts like
`npx skills add Kaelio/ktx --skill ktx` render as
`Kaelio/ktx--skill ktx`.
`npx skills add Kaelio/ktx --skill ktx` render as `Kaelio/ktx--skill ktx`.
- **MUST**: When adding a new container that renders user-visible monospace
text outside `<code>` / `<pre>` (e.g. a styled `<div className="font-mono">`
for a copyable prompt), verify the global ligature-off rule in

View file

@ -1,6 +1,6 @@
# Contributing to ktx
# Contributing to KTX
Thanks for your interest in **ktx**. This page covers **how to contribute** and
Thanks for your interest in KTX. This page covers **how to contribute** and
the **contributor rewards program**. For development setup, repository
layout, and verification commands, see the
[Contributing guide in the docs](https://docs.kaelio.com/ktx/docs/community/contributing).
@ -23,7 +23,7 @@ layout, and verification commands, see the
## Contributor rewards program
We send merch to contributors whose pull requests get merged. The goal is
to thank the people building **ktx** with us, not to drive volume.
to thank the people building KTX with us, not to drive volume.
### How it works
@ -76,7 +76,7 @@ See the [Community & Support](https://docs.kaelio.com/ktx/docs/community/support
page for the full guide. The short version:
- **Questions, "how do I...", setup help, sharing patterns**: join the
[**ktx** Slack](https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ).
[KTX Slack](https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ).
- **Bugs**: use the [Bug report](.github/ISSUE_TEMPLATE/bug_report.yml)
template.
- **Feature requests**: use the
@ -87,7 +87,7 @@ page for the full guide. The short version:
## Code of conduct
**ktx** follows the
KTX follows the
[Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/).
Be respectful, assume good intent, and keep discussion focused on the
project. Report concerns to the maintainers in Slack or by email at

View file

@ -13,20 +13,16 @@
<a href="https://docs.kaelio.com/ktx/docs/"><img src="https://img.shields.io/badge/docs-ktx-22c55e?style=flat-square" alt="Documentation" /></a>
<a href="https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ"><img src="https://img.shields.io/badge/slack-join%20community-4A154B?style=flat-square&logo=slack&logoColor=white" alt="Join the ktx Slack community" /></a>
<a href="https://github.com/Kaelio/ktx/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-Apache%202.0-blue?style=flat-square" alt="License" /></a>
<a href="https://www.ycombinator.com/companies/kaelio"><img src="https://img.shields.io/badge/Y%20Combinator-P25-orange?style=flat-square" alt="Y Combinator P25" /></a>
<a href="https://www.ycombinator.com/companies?batch=P25"><img src="https://img.shields.io/badge/Y%20Combinator-P25-orange?style=flat-square" alt="Y Combinator P25" /></a>
</p>
<p align="center">
<a href="https://docs.kaelio.com/ktx/docs/getting-started/quickstart"><b>Quickstart</b></a> ·
<a href="https://docs.kaelio.com/ktx/docs/cli-reference/ktx"><b>CLI Reference</b></a> ·
<a href="https://docs.kaelio.com/ktx/docs/community/ai-resources"><b>Agent Setup</b></a> ·
<a href="https://docs.kaelio.com/ktx/docs/ai-resources/agent-quickstart"><b>Agent Setup</b></a> ·
<a href="https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ"><b>Slack</b></a>
</p>
<p align="center">
<sub>Built and maintained by <a href="https://www.kaelio.com"><b>Kaelio</b></a></sub>
</p>
---
**ktx** is a self-improving context layer that teaches agents how to query your
@ -34,25 +30,13 @@ warehouse accurately - from approved metric definitions, joinable columns, and
business knowledge it builds and maintains for you.
> [!NOTE]
> Run **ktx** with your own LLM API keys or a local agent sign-in — a
> **Claude Pro/Max** subscription through Claude Code, or your local Codex
> authentication. No extra usage billing from **ktx**.
> Run **ktx** with your own LLM API keys or a **Claude Pro/Max** subscription.
> No extra usage billing from **ktx**.
<p align="center">
<a href="https://youtu.be/5V4TuzYVlrA">
<img src="assets/launch-video-thumb.png" alt="Watch the ktx launch video (1:56)" width="820" />
</a>
<img src="docs-site/public/images/ingestion-flow-transparent.svg" alt="ktx ingestion flow from source systems through validation to wiki and semantic-layer outputs" width="900" />
</p>
<p align="center">
<img src="docs-site/public/images/ingestion-flow.png" alt="Ingestion: ktx ingests databases, BI tools, modeling code, and docs through its context engine (source connectors, context builder, reconciliation, validation) into wiki Markdown and semantic-layer YAML" width="900" />
</p>
<p align="center">
<img src="docs-site/public/images/mcp-runtime-flow.png" alt="Serving: an agent queries ktx through MCP, which searches the wiki and semantic layer, returns approved metrics, and compiles them into read-only SQL run against the warehouse" width="900" />
</p>
## Why ktx
General-purpose agents struggle on data tasks. They re-explore your warehouse
@ -103,9 +87,8 @@ upkeep and don't absorb the rest of your company's knowledge.
- You don't have a SQL warehouse - **ktx** sits on top of one
- You only need one ad-hoc query - `psql` or a notebook will do
Works with PostgreSQL, Snowflake, BigQuery, ClickHouse, MySQL, SQL Server,
SQLite, DuckDB, Amazon Athena, and MongoDB. Integrates with dbt, MetricFlow,
LookML, Looker, Metabase, Sigma, Notion, and Google Drive.
Works with PostgreSQL, Snowflake, BigQuery, ClickHouse, MySQL, SQL Server, and
SQLite. Integrates with dbt, MetricFlow, LookML, Looker, Metabase, and Notion.
## Quick Start
@ -144,14 +127,6 @@ Agent integration ready: yes (codex:project)
> If `ktx status` prints `ktx mcp start --project-dir ...`, run it before
> opening your agent client.
## Upgrading
Re-run the global install with the `@latest` tag:
```bash
npm install -g @kaelio/ktx@latest
```
## First commands
| Command | Purpose |
@ -189,9 +164,8 @@ then the current directory. Pass `--project-dir <path>` when scripting.
No. **ktx** runs locally. The only data leaving your machine is what you
send to the LLM provider you configured.
- **Which LLM backends are supported?**
Anthropic API, Google Vertex AI, AI Gateway, the local Claude Code session
through the Claude Agent SDK, and your local Codex authentication through the
Codex SDK. See
Anthropic API, Google Vertex AI, AI Gateway, and the local Claude Code
session through the Claude Agent SDK. See
[LLM configuration](https://docs.kaelio.com/ktx/docs/guides/llm-configuration).
- **How is ktx different from a dbt or MetricFlow semantic layer?**
**ktx** *ingests* those layers and combines them with raw-table
@ -210,7 +184,7 @@ then the current directory. Pass `--project-dir <path>` when scripting.
- [The Context Layer](https://docs.kaelio.com/ktx/docs/concepts/the-context-layer)
- [Building Context](https://docs.kaelio.com/ktx/docs/guides/building-context)
- [CLI Reference](https://docs.kaelio.com/ktx/docs/cli-reference/ktx)
- [AI Resources](https://docs.kaelio.com/ktx/docs/community/ai-resources)
- [Agent Quickstart](https://docs.kaelio.com/ktx/docs/ai-resources/agent-quickstart)
- [Community & Support](https://docs.kaelio.com/ktx/docs/community/support)
## Community
@ -260,17 +234,11 @@ uv run pytest -q
## Telemetry
**ktx** collects privacy-conscious usage telemetry to understand installs and
improve setup, command reliability, and data-agent workflows. Catalog telemetry
events do not record file paths, hostnames, SQL, schema names, table names,
column names, error messages, raw environment values, or argv. Error reports use
PostHog Error Tracking and can include stack frames and raw error messages,
which may contain local file paths or the local username in those paths.
**ktx** redacts secrets, credentials, database URLs, auth headers, argv, raw
environment values, SQL text, row data, and user-typed prompt or MCP argument
text from the explicit `$exception` payload. See
[Telemetry](https://docs.kaelio.com/ktx/docs/community/telemetry) for the event
catalog and opt-out options.
**ktx** collects anonymous usage telemetry from interactive CLI runs to
improve setup, command reliability, and data-agent workflows. No file paths,
hostnames, SQL, schema names, error messages, or argv are recorded. See
[Telemetry](https://docs.kaelio.com/ktx/docs/community/telemetry) for the
event catalog and opt-out options.
## License
@ -280,6 +248,6 @@ catalog and opt-out options.
<p align="center">
<a href="https://star-history.com/#Kaelio/ktx&Date">
<img src="assets/star-history.svg" alt="ktx Star History Chart" width="700" />
<img src="https://api.star-history.com/svg?repos=Kaelio/ktx&type=Date" alt="ktx Star History Chart" width="700" />
</a>
</p>

View file

@ -2,20 +2,20 @@
## Reporting a vulnerability
If you believe you've found a security vulnerability in **ktx**, please report it
If you believe you've found a security vulnerability in KTX, please report it
**privately** through GitHub Security Advisories:
[Report a vulnerability](https://github.com/Kaelio/ktx/security/advisories/new)
If you cannot use GitHub Security Advisories, email `support@kaelio.com`
instead. Please do **not** open a public issue, post in the **ktx** Slack, or
instead. Please do **not** open a public issue, post in the KTX Slack, or
share details elsewhere until we have published a fix.
When reporting, please include:
- A description of the issue and its impact
- Steps to reproduce
- The **ktx** version affected
- The KTX version affected
## What to expect

View file

@ -19,9 +19,14 @@
<path d="M 80 84 Q 86 77 92 84" fill="none" stroke="#F5F1EA" stroke-width="3.5" stroke-linecap="round" />
<path d="M 108 84 Q 114 77 120 84" fill="none" stroke="#F5F1EA" stroke-width="3.5" stroke-linecap="round" />
<!-- wordmark: "ktx" outlined from Outfit SemiBold (the docs-site display font)
so it renders identically everywhere, independent of installed fonts -->
<g transform="translate(242 145)" fill="#1B3139">
<path d="M51.17 0 25.06 -34.79 51.03 -67.62H72.17L41.65 -30.7L42.35 -39.55L73.57 0ZM8.05 0V-101.22H26.46V0ZM88.41 0V-95.69H106.82V0ZM72.66 -51.52V-67.62H122.57V-51.52ZM171.75 0 153.93 -27.41 150.22 -30.17 123.83 -67.62H145.64L161.91 -42.77L165.48 -40.18L193.38 0ZM122.54 0 150.05 -38.61 160.62 -26.22 143.19 0ZM166.11 -30.38 155.44 -42.67 171.54 -67.62H192.08Z" />
</g>
<!-- wordmark: 'ktx', half the logo height, vertically centered -->
<text
x="225"
y="145"
font-family="'JetBrains Mono', 'Fira Code', ui-monospace, 'SF Mono', Menlo, monospace"
font-size="140"
font-weight="600"
fill="#1B3139"
letter-spacing="-0.04em"
>ktx</text>
</svg>

Before

Width:  |  Height:  |  Size: 1.4 KiB

After

Width:  |  Height:  |  Size: 1.1 KiB

Before After
Before After

Binary file not shown.

Before

Width:  |  Height:  |  Size: 135 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 64 KiB

View file

@ -1,12 +0,0 @@
import type { Metadata } from "next";
import { DiagramStudio } from "@/components/diagram-studio/studio";
export const metadata: Metadata = {
title: "Diagram studio",
robots: { index: false, follow: false },
};
export default function DiagramStudioPage() {
return <DiagramStudio />;
}

View file

@ -869,97 +869,6 @@ body::after {
50% { opacity: 0.65; transform: scale(0.9); }
}
/*
GitHub star widget (sidebar footer pill)
Rendered as the `icon` of a fumadocs icon-link, so it sits in the footer
pill beside the Slack mark and the theme toggle. GitHub mark + star glyph
+ live count; the star rotates to coral on hover. The !important sizes win
over fumadocs' `[&_svg]:size-4.5` rule on the wrapping link.
*/
.ktx-stars {
display: inline-flex;
align-items: center;
gap: 6px;
font-family: var(--font-display), var(--font-sans), sans-serif;
font-size: 13px;
line-height: 1;
}
/* Push the stars to the opposite (right) end of the footer pill, leaving the
Slack mark on the left like justify-content: space-between. The auto margin
absorbs the pill's free space; we cancel the theme toggle's own ms-auto so
that single gap lands before the stars, not between stars and the toggle. */
#nd-sidebar a[aria-label="Star ktx on GitHub"] {
margin-inline-start: auto;
}
#nd-sidebar [data-theme-toggle] {
margin-inline-start: 0;
}
.ktx-stars-gh {
width: 16px !important;
height: 16px !important;
flex-shrink: 0;
}
.ktx-stars-count-wrap {
display: inline-flex;
align-items: center;
gap: 4px;
}
.ktx-stars-star {
width: 12px !important;
height: 12px !important;
flex-shrink: 0;
fill: currentColor;
opacity: 0.7;
transition:
transform 0.3s var(--ktx-ease),
fill 0.3s var(--ktx-ease),
opacity 0.3s var(--ktx-ease);
}
/* The wrapping fumadocs link owns the hover; rotate + colour the star from it. */
#nd-sidebar a:hover .ktx-stars-star {
transform: rotate(-14deg) scale(1.12);
fill: var(--ktx-coral);
opacity: 1;
}
.ktx-stars-count {
font-weight: 600;
font-variant-numeric: tabular-nums;
letter-spacing: -0.01em;
}
/* Skeleton shown only on the rare cold (uncached) fetch */
.ktx-stars-skeleton-bar {
display: inline-block;
width: 26px;
height: 10px;
border-radius: 4px;
background: linear-gradient(
90deg,
var(--color-fd-muted) 25%,
color-mix(in oklch, var(--color-fd-muted-foreground) 28%, var(--color-fd-muted)) 50%,
var(--color-fd-muted) 75%
);
background-size: 200% 100%;
animation: ktx-stars-shimmer 1.4s ease-in-out infinite;
}
@keyframes ktx-stars-shimmer {
from { background-position: 200% 0; }
to { background-position: -200% 0; }
}
@media (prefers-reduced-motion: reduce) {
#nd-sidebar a:hover .ktx-stars-star { transform: none; }
.ktx-stars-skeleton-bar { animation: none; }
}
/* Dot grid */
.dot-grid {
background-image: radial-gradient(

View file

@ -1,22 +1,22 @@
import type { BaseLayoutProps } from "fumadocs-ui/layouts/shared";
import { GitHubIcon } from "@/components/github-icon";
import { Logo } from "@/components/logo";
import { SlackIcon } from "@/components/slack-icon";
import { GitHubStars, GITHUB_REPO_URL } from "@/components/github-stars";
import { ThemeToggle } from "@/components/theme-toggle";
export const baseOptions: BaseLayoutProps = {
nav: {
title: Logo,
title: <Logo />,
transparentMode: "top",
},
// Custom two-icon switcher (light / dark) where each icon selects its own
// theme. The default "light-dark" switcher is a single blind toggle — both
// icons just flip the theme, so clicking the sun while already in light mode
// jumps to dark, which reads as broken.
slots: {
themeSwitch: ThemeToggle,
},
links: [
{
type: "icon",
label: "GitHub",
icon: <GitHubIcon />,
text: "GitHub",
url: "https://github.com/kaelio/ktx",
external: true,
},
{
type: "icon",
label: "Join the ktx Slack community",
@ -25,13 +25,5 @@ export const baseOptions: BaseLayoutProps = {
url: "https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ",
external: true,
},
{
type: "icon",
label: "Star ktx on GitHub",
icon: <GitHubStars />,
text: "GitHub",
url: GITHUB_REPO_URL,
external: true,
},
],
};

View file

@ -1,328 +0,0 @@
import { type Edge, MarkerType, type Node } from "@xyflow/react";
import { C } from "./nodes";
const EDGE_COLOR = "#b3bcc4";
const MARKER_COLOR = "#9aa6ad";
const labelStyle = {
fontFamily: "var(--font-inter), system-ui, sans-serif",
fontSize: 15,
fontWeight: 600,
fill: C.inkMuted,
};
const labelBgStyle = { fill: "#ffffff", stroke: C.chipBorder, strokeWidth: 1 };
const labelBg = {
labelBgPadding: [8, 4] as [number, number],
labelBgBorderRadius: 6,
labelStyle,
labelBgStyle,
};
const marker = { type: MarkerType.ArrowClosed, color: MARKER_COLOR, width: 16, height: 16 };
const edgeStyle = { stroke: EDGE_COLOR, strokeWidth: 2 };
/* ============================== INGESTION =============================== */
const SRC_W = 300;
const SRC_H = 138;
const SRC_GAP = 24;
const srcY = (i: number) => i * (SRC_H + SRC_GAP);
export const ingestionNodes: Node[] = [
{
id: "title",
type: "title",
position: { x: 0, y: -96 },
data: {
width: 560,
eyebrow: "1 · Ingestion",
title: "ktx builds your context layer",
},
},
{
id: "db",
type: "card",
position: { x: 0, y: srcY(0) },
data: {
width: SRC_W,
height: SRC_H,
accent: C.teal,
rows: [
{ kind: "title", text: "Databases" },
{ kind: "desc", text: "Schemas, keys, query history" },
{ kind: "muted", text: "Postgres · Snowflake · BigQuery · …" },
],
handles: [{ side: "right", type: "source", id: "out" }],
},
},
{
id: "bi",
type: "card",
position: { x: 0, y: srcY(1) },
data: {
width: SRC_W,
height: SRC_H,
accent: C.orange,
rows: [
{ kind: "title", text: "BI tools" },
{ kind: "desc", text: "Dashboards, explores, usage" },
{ kind: "muted", text: "Metabase · Looker · …" },
],
handles: [{ side: "right", type: "source", id: "out" }],
},
},
{
id: "model",
type: "card",
position: { x: 0, y: srcY(2) },
data: {
width: SRC_W,
height: SRC_H,
accent: C.amber,
rows: [
{ kind: "title", text: "Modeling code" },
{ kind: "desc", text: "Metrics, models, joins, entities" },
{ kind: "muted", text: "dbt · LookML · MetricFlow · …" },
],
handles: [{ side: "right", type: "source", id: "out" }],
},
},
{
id: "docs",
type: "card",
position: { x: 0, y: srcY(3) },
data: {
width: SRC_W,
height: SRC_H,
accent: C.emerald,
rows: [
{ kind: "title", text: "Docs & notes" },
{ kind: "desc", text: "Policies, definitions, notes" },
{ kind: "muted", text: "Notion · any text · …" },
],
handles: [{ side: "right", type: "source", id: "out" }],
},
},
{
id: "engine",
type: "engine",
position: { x: 420, y: 52 },
data: {
width: 380,
height: 520,
steps: [
{ n: 1, title: "Source connectors", desc: "Read each source in its shape" },
{ n: 2, title: "Context builder", desc: "Evidence into proposed updates" },
{ n: 3, title: "Reconciliation", desc: "Merge with existing context" },
{ n: 4, title: "Validation", desc: "Check references & semantics" },
],
handles: [
{ side: "left", type: "target", id: "in" },
{ side: "right", type: "source", id: "out" },
],
},
},
{
id: "wiki",
type: "card",
position: { x: 900, y: 66 },
data: {
width: 320,
height: 220,
accent: C.emerald,
rows: [
{ kind: "mono", text: "wiki/*.md", color: C.emerald },
{ kind: "title", text: "Wiki" },
{ kind: "chips", items: ["free-form", "auto-maintained"] },
{ kind: "desc", text: "Definitions, caveats, policies," },
{ kind: "desc", text: "and notes agents can search." },
],
handles: [{ side: "left", type: "target", id: "in" }],
},
},
{
id: "sl",
type: "card",
position: { x: 900, y: 338 },
data: {
width: 320,
height: 220,
accent: C.teal,
rows: [
{ kind: "mono", text: "semantic-layer/*.yaml", color: C.teal },
{ kind: "title", text: "Semantic layer" },
{ kind: "chips", items: ["executable", "auto-maintained"] },
{ kind: "desc", text: "Metrics, joins, dimensions, and" },
{ kind: "desc", text: "filters ktx compiles into SQL." },
],
handles: [{ side: "left", type: "target", id: "in" }],
},
},
];
const ingestEdge = (source: string, target: string): Edge => ({
id: `${source}-${target}`,
source,
target,
sourceHandle: "out",
targetHandle: "in",
type: "default",
style: edgeStyle,
markerEnd: marker,
});
export const ingestionEdges: Edge[] = [
ingestEdge("db", "engine"),
ingestEdge("bi", "engine"),
ingestEdge("model", "engine"),
ingestEdge("docs", "engine"),
ingestEdge("engine", "wiki"),
ingestEdge("engine", "sl"),
];
/* =============================== RUNTIME ================================ */
export const runtimeNodes: Node[] = [
{
id: "title",
type: "title",
position: { x: 0, y: -84 },
data: {
width: 560,
eyebrow: "2 · Serving",
title: "agents query it through MCP",
},
},
{
id: "agent",
type: "card",
position: { x: 0, y: 115 },
data: {
width: 280,
height: 190,
accent: C.neutral,
align: "center",
rows: [
{ kind: "title", text: "Your agent" },
{ kind: "muted", text: "Claude Code · Cursor" },
{ kind: "muted", text: "Codex · OpenCode" },
],
handles: [
{ side: "right", type: "source", id: "ask", top: "42%" },
{ side: "right", type: "target", id: "answer", top: "62%" },
],
},
},
{
id: "hub",
type: "hub",
position: { x: 420, y: 85 },
data: {
width: 360,
height: 250,
rows: [
"Search wiki + semantic layer",
"Return approved metrics",
"Compile metrics → SQL",
],
handles: [
{ side: "left", type: "target", id: "ask", top: "42%" },
{ side: "left", type: "source", id: "answer", top: "62%" },
{ side: "right", type: "source", id: "to-context", top: "30%" },
{ side: "right", type: "source", id: "to-warehouse", top: "72%" },
],
},
},
{
id: "context",
type: "card",
position: { x: 920, y: 15 },
data: {
width: 300,
height: 150,
accent: C.teal,
rows: [
{ kind: "title", text: "Context layer" },
{ kind: "mono", text: "wiki/*.md", color: C.emerald },
{ kind: "mono", text: "semantic-layer/*.yaml", color: C.teal },
],
handles: [{ side: "left", type: "target", id: "in" }],
},
},
{
id: "warehouse",
type: "card",
position: { x: 920, y: 255 },
data: {
width: 300,
height: 150,
accent: C.slate,
rows: [
{ kind: "title", text: "Warehouse" },
{
kind: "badge",
text: "read-only",
bg: "#ecf6f8",
border: "#bfe3ea",
color: C.teal,
},
{ kind: "desc", text: "Runs the compiled SQL" },
],
handles: [{ side: "left", type: "target", id: "in" }],
},
},
];
export const runtimeEdges: Edge[] = [
{
id: "ask",
source: "agent",
sourceHandle: "ask",
target: "hub",
targetHandle: "ask",
type: "default",
label: "ask",
...labelBg,
style: edgeStyle,
markerEnd: marker,
},
{
id: "answer",
source: "hub",
sourceHandle: "answer",
target: "agent",
targetHandle: "answer",
type: "default",
label: "answer",
...labelBg,
style: edgeStyle,
markerEnd: marker,
},
{
id: "search",
source: "hub",
sourceHandle: "to-context",
target: "context",
targetHandle: "in",
type: "smoothstep",
label: "search + read",
...labelBg,
style: edgeStyle,
markerStart: marker,
markerEnd: marker,
},
{
id: "readonly",
source: "hub",
sourceHandle: "to-warehouse",
target: "warehouse",
targetHandle: "in",
type: "smoothstep",
label: "read-only",
...labelBg,
style: edgeStyle,
markerStart: marker,
markerEnd: marker,
},
];

View file

@ -1,57 +0,0 @@
/**
* Inlined ktx mascot, ported from assets/ktx-mascot.svg.
*
* - `light` renders the dark-bodied mascot for light surfaces.
* - `dark` renders the cream-bodied mascot for dark surfaces (e.g. the ktx
* hub panel), mirroring brand/ktx-mascot-dark.svg.
*/
export function KtxMascot({
variant = "light",
size = 56,
}: {
variant?: "light" | "dark";
size?: number;
}) {
const body = variant === "dark" ? "#F5F1EA" : "#1B3139";
const eye = variant === "dark" ? "#1B3139" : "#F5F1EA";
return (
<svg
viewBox="0 0 200 200"
width={size}
height={size}
role="img"
aria-label="ktx mascot"
>
<g fill="none" stroke={body} strokeWidth="16" strokeLinecap="round">
<path d="M 62 110 Q 32 130 44 152" />
<path d="M 88 116 Q 80 152 70 174" />
<path d="M 112 116 Q 120 152 130 174" />
</g>
<path
d="M 134 108 C 162 116, 172 96, 162 78 C 154 64, 168 56, 178 60"
fill="none"
stroke="#FF8A4C"
strokeWidth="16"
strokeLinecap="round"
/>
<path
d="M 48 102 C 48 56, 78 30, 100 30 C 122 30, 152 56, 152 102 C 152 116, 132 120, 100 120 C 68 120, 48 116, 48 102 Z"
fill={body}
/>
<path
d="M 80 84 Q 86 77 92 84"
fill="none"
stroke={eye}
strokeWidth="3.5"
strokeLinecap="round"
/>
<path
d="M 108 84 Q 114 77 120 84"
fill="none"
stroke={eye}
strokeWidth="3.5"
strokeLinecap="round"
/>
</svg>
);
}

View file

@ -1,493 +0,0 @@
"use client";
import { Handle, Position, type Node, type NodeProps } from "@xyflow/react";
import { KtxMascot } from "./mascot";
/** Fixed palette mirrored from the approved SVG diagrams so the exported PNG
* is theme-independent (one image that reads on light and dark GitHub). */
export const C = {
ink: "#1b1b18",
inkSoft: "#57534e",
inkMuted: "#8c857f",
cardBorder: "#e2dfd9",
engineBg: "#15323a",
engineBorder: "#23474f",
cyan: "#55dced",
stepNum: "#06262c",
stepTitle: "#f3f1ec",
stepDesc: "#9fb6bc",
hubRow: "#eef4f5",
chipBg: "#faf9f6",
chipBorder: "#e7e5e4",
teal: "#0e7490",
emerald: "#059669",
orange: "#f97316",
amber: "#d97706",
slate: "#334155",
neutral: "#94a3b8",
} as const;
const DISPLAY = "var(--font-display), system-ui, sans-serif";
const BODY = "var(--font-inter), system-ui, sans-serif";
const MONO = "var(--font-mono), ui-monospace, monospace";
const CARD_SHADOW = "0 3px 12px rgba(27, 49, 57, 0.10)";
const ENGINE_SHADOW = "0 6px 22px rgba(2, 12, 15, 0.30)";
/** ktx logo mascot size, shared by the engine and hub headers. */
const LOGO_SIZE = 56;
type HandleSpec = {
side: "left" | "right";
type: "source" | "target";
id: string;
top?: string;
};
function Handles({ specs }: { specs?: HandleSpec[] }) {
if (!specs) return null;
return (
<>
{specs.map((h) => (
<Handle
key={`${h.type}-${h.id}`}
id={h.id}
type={h.type}
position={h.side === "left" ? Position.Left : Position.Right}
isConnectable={false}
style={{
opacity: 0,
border: 0,
background: "transparent",
...(h.top ? { top: h.top } : {}),
}}
/>
))}
</>
);
}
/* ------------------------------- Card node ------------------------------- */
type CardRow =
| { kind: "title"; text: string }
| { kind: "mono"; text: string; color: string }
| { kind: "desc"; text: string }
| { kind: "muted"; text: string }
| { kind: "chips"; items: string[] }
| { kind: "badge"; text: string; bg: string; border: string; color: string };
type CardData = {
width: number;
height: number;
accent: string;
align?: "center";
rows: CardRow[];
handles?: HandleSpec[];
};
function gapFor(kind: CardRow["kind"], prev?: CardRow["kind"]): number {
if (!prev) return 0;
if (kind === "desc" && prev === "desc") return 3;
if (kind === "mono" && prev === "mono") return 2;
if (kind === "title") return 6;
return 10;
}
function CardRowView({ row }: { row: CardRow }) {
switch (row.kind) {
case "title":
return (
<span
style={{
fontFamily: DISPLAY,
fontWeight: 700,
fontSize: 26,
lineHeight: 1.15,
color: C.ink,
}}
>
{row.text}
</span>
);
case "mono":
return (
<span
style={{
fontFamily: MONO,
fontWeight: 700,
fontSize: 18,
lineHeight: 1.4,
color: row.color,
}}
>
{row.text}
</span>
);
case "desc":
return (
<span
style={{
fontFamily: BODY,
fontWeight: 500,
fontSize: 17,
lineHeight: 1.45,
color: C.inkSoft,
}}
>
{row.text}
</span>
);
case "muted":
return (
<span
style={{
fontFamily: BODY,
fontWeight: 500,
fontSize: 14,
lineHeight: 1.4,
color: C.inkMuted,
}}
>
{row.text}
</span>
);
case "chips":
return (
<div style={{ display: "flex", gap: 8, flexWrap: "wrap" }}>
{row.items.map((c) => (
<span
key={c}
style={{
fontFamily: BODY,
fontWeight: 600,
fontSize: 14,
color: C.inkSoft,
background: C.chipBg,
border: `1px solid ${C.chipBorder}`,
borderRadius: 6,
padding: "4px 10px",
}}
>
{c}
</span>
))}
</div>
);
case "badge":
return (
<span
style={{
display: "inline-flex",
alignItems: "center",
borderRadius: 14,
padding: "3px 12px",
fontFamily: BODY,
fontWeight: 700,
fontSize: 14,
background: row.bg,
border: `1px solid ${row.border}`,
color: row.color,
}}
>
{row.text}
</span>
);
}
}
function CardNode({ data }: NodeProps<Node<CardData>>) {
const center = data.align === "center";
return (
<div
style={{
width: data.width,
height: data.height,
position: "relative",
background: "#ffffff",
border: `1px solid ${C.cardBorder}`,
borderRadius: 10,
boxShadow: CARD_SHADOW,
padding: "18px 20px",
display: "flex",
flexDirection: "column",
alignItems: center ? "center" : "flex-start",
justifyContent: center ? "center" : "flex-start",
textAlign: center ? "center" : "left",
overflow: "hidden",
}}
>
<span
style={{
position: "absolute",
top: 0,
left: 2,
right: 2,
height: 4,
borderRadius: 2,
background: data.accent,
}}
/>
<Handles specs={data.handles} />
{data.rows.map((row, i) => (
<div
key={i}
style={{ marginTop: gapFor(row.kind, data.rows[i - 1]?.kind) }}
>
<CardRowView row={row} />
</div>
))}
</div>
);
}
/* ------------------------------ Engine node ------------------------------ */
type EngineStep = { n: number; title: string; desc: string };
type EngineData = {
width: number;
height: number;
steps: EngineStep[];
handles?: HandleSpec[];
};
function EngineNode({ data }: NodeProps<Node<EngineData>>) {
return (
<div
style={{
width: data.width,
height: data.height,
position: "relative",
background: C.engineBg,
border: `1px solid ${C.engineBorder}`,
borderRadius: 14,
boxShadow: ENGINE_SHADOW,
padding: "24px 24px",
display: "flex",
flexDirection: "column",
overflow: "hidden",
}}
>
<span
style={{
position: "absolute",
top: 0,
left: 2,
right: 2,
height: 4,
borderRadius: 2,
background: C.cyan,
}}
/>
<Handles specs={data.handles} />
<div style={{ display: "flex", alignItems: "center", gap: 14 }}>
<KtxMascot variant="dark" size={LOGO_SIZE} />
<span
style={{
fontFamily: DISPLAY,
fontWeight: 700,
fontSize: 30,
color: C.stepTitle,
}}
>
ktx
</span>
</div>
<div
style={{
flex: 1,
display: "flex",
flexDirection: "column",
justifyContent: "space-around",
marginTop: 6,
}}
>
{data.steps.map((s) => (
<div
key={s.n}
style={{ display: "flex", alignItems: "center", gap: 18 }}
>
<span
style={{
flex: "none",
width: 44,
height: 44,
borderRadius: "50%",
background: C.cyan,
display: "flex",
alignItems: "center",
justifyContent: "center",
fontFamily: DISPLAY,
fontWeight: 800,
fontSize: 22,
color: C.stepNum,
}}
>
{s.n}
</span>
<div style={{ display: "flex", flexDirection: "column", gap: 3 }}>
<span
style={{
fontFamily: DISPLAY,
fontWeight: 700,
fontSize: 24,
lineHeight: 1.1,
color: C.stepTitle,
}}
>
{s.title}
</span>
<span
style={{
fontFamily: BODY,
fontWeight: 500,
fontSize: 16,
lineHeight: 1.3,
color: C.stepDesc,
}}
>
{s.desc}
</span>
</div>
</div>
))}
</div>
</div>
);
}
/* -------------------------------- Hub node ------------------------------- */
type HubData = {
width: number;
height: number;
rows: string[];
handles?: HandleSpec[];
};
function HubNode({ data }: NodeProps<Node<HubData>>) {
return (
<div
style={{
width: data.width,
height: data.height,
position: "relative",
background: C.engineBg,
border: `1px solid ${C.engineBorder}`,
borderRadius: 14,
boxShadow: ENGINE_SHADOW,
padding: "24px 24px",
display: "flex",
flexDirection: "column",
overflow: "hidden",
}}
>
<span
style={{
position: "absolute",
top: 0,
left: 2,
right: 2,
height: 4,
borderRadius: 2,
background: C.cyan,
}}
/>
<Handles specs={data.handles} />
<div style={{ display: "flex", alignItems: "center", gap: 14 }}>
<KtxMascot variant="dark" size={LOGO_SIZE} />
<span
style={{
fontFamily: DISPLAY,
fontWeight: 700,
fontSize: 30,
color: C.stepTitle,
}}
>
ktx
</span>
</div>
<div
style={{
marginTop: 22,
display: "flex",
flexDirection: "column",
gap: 18,
}}
>
{data.rows.map((r) => (
<div key={r} style={{ display: "flex", alignItems: "center", gap: 14 }}>
<span
style={{
flex: "none",
width: 10,
height: 10,
borderRadius: "50%",
background: C.cyan,
}}
/>
<span
style={{
fontFamily: BODY,
fontWeight: 600,
fontSize: 19,
color: C.hubRow,
}}
>
{r}
</span>
</div>
))}
</div>
</div>
);
}
/* ------------------------------- Title node ------------------------------ */
type TitleData = { width: number; eyebrow: string; title: string };
function TitleNode({ data }: NodeProps<Node<TitleData>>) {
return (
<div
style={{
width: data.width,
display: "flex",
flexDirection: "column",
gap: 6,
}}
>
<span
style={{
fontFamily: BODY,
fontSize: 19,
fontWeight: 800,
letterSpacing: 2,
textTransform: "uppercase",
color: C.teal,
}}
>
{data.eyebrow}
</span>
<span
style={{
fontFamily: DISPLAY,
fontSize: 24,
fontWeight: 600,
color: C.inkMuted,
}}
>
{data.title}
</span>
</div>
);
}
export const nodeTypes = {
card: CardNode,
engine: EngineNode,
hub: HubNode,
title: TitleNode,
};

View file

@ -1,242 +0,0 @@
"use client";
import "@xyflow/react/dist/style.css";
import { useCallback, useRef, useState } from "react";
import {
Background,
BackgroundVariant,
type Edge,
getNodesBounds,
type Node,
ReactFlow,
ReactFlowProvider,
useEdgesState,
useNodesState,
useReactFlow,
} from "@xyflow/react";
import { toPng } from "html-to-image";
import {
ingestionEdges,
ingestionNodes,
runtimeEdges,
runtimeNodes,
} from "./flows";
import { nodeTypes } from "./nodes";
const EXPORT_PADDING = 48;
const EXPORT_PIXEL_RATIO = 2;
function DiagramCanvasInner({
initialNodes,
initialEdges,
fileName,
height,
dark,
}: {
initialNodes: Node[];
initialEdges: Edge[];
fileName: string;
height: number;
dark: boolean;
}) {
const wrapperRef = useRef<HTMLDivElement>(null);
const [nodes, , onNodesChange] = useNodesState(initialNodes);
const [edges, , onEdgesChange] = useEdgesState(initialEdges);
const { getNodes } = useReactFlow();
const [busy, setBusy] = useState(false);
const download = useCallback(async () => {
const viewport = wrapperRef.current?.querySelector<HTMLElement>(
".react-flow__viewport",
);
if (!viewport) return;
setBusy(true);
try {
await document.fonts.ready;
const bounds = getNodesBounds(getNodes());
const outW = Math.ceil(bounds.width + EXPORT_PADDING * 2);
const outH = Math.ceil(bounds.height + EXPORT_PADDING * 2);
const tx = EXPORT_PADDING - bounds.x;
const ty = EXPORT_PADDING - bounds.y;
const dataUrl = await toPng(viewport, {
width: outW,
height: outH,
pixelRatio: EXPORT_PIXEL_RATIO,
// transparent background so one PNG works on light and dark GitHub
style: {
width: `${outW}px`,
height: `${outH}px`,
transform: `translate(${tx}px, ${ty}px) scale(1)`,
},
});
const link = document.createElement("a");
link.download = fileName;
link.href = dataUrl;
link.click();
} finally {
setBusy(false);
}
}, [fileName, getNodes]);
return (
<div>
<div style={{ display: "flex", gap: 8, marginBottom: 10 }}>
<button
type="button"
onClick={download}
disabled={busy}
style={btnStyle(busy)}
>
{busy ? "Exporting…" : "Download PNG"}
</button>
</div>
<div
ref={wrapperRef}
style={{
height,
borderRadius: 12,
border: "1px solid rgba(127,127,127,0.2)",
background: dark ? "#0d1117" : "#ffffff",
}}
>
<ReactFlow
nodes={nodes}
edges={edges}
nodeTypes={nodeTypes}
onNodesChange={onNodesChange}
onEdgesChange={onEdgesChange}
fitView
fitViewOptions={{ padding: 0.08 }}
nodesDraggable={false}
nodesConnectable={false}
nodesFocusable={false}
edgesFocusable={false}
elementsSelectable={false}
panOnDrag={false}
panOnScroll={false}
zoomOnScroll={false}
zoomOnPinch={false}
zoomOnDoubleClick={false}
preventScrolling={false}
proOptions={{ hideAttribution: true }}
>
<Background
variant={BackgroundVariant.Dots}
gap={18}
size={1}
color={dark ? "#1f2a30" : "#e6e2db"}
/>
</ReactFlow>
</div>
</div>
);
}
function btnStyle(disabled: boolean): React.CSSProperties {
return {
fontFamily: "var(--font-inter), system-ui, sans-serif",
fontSize: 13,
fontWeight: 600,
padding: "7px 14px",
borderRadius: 8,
border: "1px solid #0e7490",
background: disabled ? "#9bbdc6" : "#0e7490",
color: "#ffffff",
cursor: disabled ? "default" : "pointer",
};
}
function DiagramCanvas(props: {
initialNodes: Node[];
initialEdges: Edge[];
fileName: string;
height: number;
dark: boolean;
}) {
return (
<ReactFlowProvider>
<DiagramCanvasInner {...props} />
</ReactFlowProvider>
);
}
export function DiagramStudio() {
const [dark, setDark] = useState(false);
return (
<main
style={{
maxWidth: 1320,
margin: "0 auto",
padding: "32px 24px 80px",
fontFamily: "var(--font-inter), system-ui, sans-serif",
}}
>
<header style={{ marginBottom: 24 }}>
<h1
style={{
fontFamily: "var(--font-display), system-ui, sans-serif",
fontSize: 30,
fontWeight: 700,
color: "#1b1b18",
margin: 0,
}}
>
ktx diagram studio
</h1>
<p style={{ color: "#6b6560", marginTop: 6, fontSize: 15 }}>
Static diagrams. Export is a transparent 2× PNG framed to the node
bounds the dark-background toggle is only for previewing.
</p>
<label
style={{
display: "inline-flex",
alignItems: "center",
gap: 8,
marginTop: 12,
fontSize: 14,
color: "#57534e",
}}
>
<input
type="checkbox"
checked={dark}
onChange={(e) => setDark(e.target.checked)}
/>
Preview on dark background
</label>
</header>
<section style={{ marginBottom: 40 }}>
<h2 style={sectionTitle}>1 · Ingestion building the context layer</h2>
<DiagramCanvas
initialNodes={ingestionNodes}
initialEdges={ingestionEdges}
fileName="ingestion-flow.png"
height={560}
dark={dark}
/>
</section>
<section>
<h2 style={sectionTitle}>2 · Serving answering agents at runtime</h2>
<DiagramCanvas
initialNodes={runtimeNodes}
initialEdges={runtimeEdges}
fileName="mcp-runtime-flow.png"
height={480}
dark={dark}
/>
</section>
</main>
);
}
const sectionTitle: React.CSSProperties = {
fontFamily: "var(--font-display), system-ui, sans-serif",
fontSize: 18,
fontWeight: 600,
color: "#1b1b18",
marginBottom: 12,
};

View file

@ -1,82 +0,0 @@
import { Suspense } from "react";
import { GitHubIcon } from "@/components/github-icon";
const REPO = "kaelio/ktx";
export const GITHUB_REPO_URL = `https://github.com/${REPO}`;
const API_URL = `https://api.github.com/repos/${REPO}`;
async function fetchStarCount(): Promise<number | null> {
try {
const res = await fetch(API_URL, {
headers: { Accept: "application/vnd.github+json" },
// Revalidate hourly. GitHub's unauthenticated REST limit is 60 req/h per
// IP, so a single cached server-side fetch keeps the count fresh while
// never exposing visitors to rate limits or layout shift.
next: { revalidate: 3600 },
});
if (!res.ok) return null;
const data = (await res.json()) as { stargazers_count?: unknown };
return typeof data.stargazers_count === "number"
? data.stargazers_count
: null;
} catch {
return null;
}
}
/** Compact, GitHub-style count: 847 → "847", 1234 → "1.2k", 12345 → "12.3k". */
function formatStars(count: number): string {
if (count < 1000) return count.toLocaleString("en-US");
const thousands = count / 1000;
const rounded =
thousands >= 100 ? Math.round(thousands) : Math.round(thousands * 10) / 10;
return `${rounded}k`;
}
function StarGlyph() {
return (
<svg className="ktx-stars-star" viewBox="0 0 24 24" aria-hidden="true">
<path d="M12 2.6l2.9 5.88 6.49.95-4.7 4.57 1.11 6.46L12 17.4l-5.8 3.06 1.11-6.46-4.7-4.57 6.49-.95z" />
</svg>
);
}
async function StarsInner() {
const count = await fetchStarCount();
return (
<span className="ktx-stars">
<GitHubIcon className="ktx-stars-gh" />
{count !== null ? (
<span className="ktx-stars-count-wrap">
<StarGlyph />
<span className="ktx-stars-count">{formatStars(count)}</span>
</span>
) : (
<span className="ktx-stars-count">Star</span>
)}
</span>
);
}
function StarsSkeleton() {
return (
<span className="ktx-stars" aria-hidden="true">
<GitHubIcon className="ktx-stars-gh" />
<span className="ktx-stars-skeleton-bar" />
</span>
);
}
/**
* Footer star widget GitHub mark + live count. Rendered as the `icon` of a
* fumadocs `type: "icon"` link, so it lands in the sidebar footer pill beside
* the Slack icon and the theme toggle. fumadocs supplies the surrounding <a>
* (href + aria-label), so this renders inner content only no anchor.
*/
export function GitHubStars() {
return (
<Suspense fallback={<StarsSkeleton />}>
<StarsInner />
</Suspense>
);
}

View file

@ -1,56 +1,40 @@
"use client";
import Link from "next/link";
const brandFont = {
fontFamily: "var(--font-display), var(--font-sans), sans-serif",
} as const;
export function Logo({ href = "/", className }: { href?: string; className?: string }) {
export function Logo() {
return (
<div className={className}>
<div className="flex items-center gap-3.5 group">
<Link href={href} aria-label="ktx documentation home" className="flex items-center no-underline">
<span className="relative flex items-center justify-center transition-transform duration-300 ease-out group-hover:rotate-[-4deg]">
<img
src="/ktx/brand/ktx-mascot.svg"
alt=""
aria-hidden="true"
className="h-20 w-20 object-contain block dark:hidden"
/>
<img
src="/ktx/brand/ktx-mascot-dark.svg"
alt=""
aria-hidden="true"
className="h-20 w-20 object-contain hidden dark:block"
/>
</span>
</Link>
<div className="flex flex-col items-start leading-none">
<Link
href={href}
className="text-[42px] font-semibold text-fd-foreground tracking-tight no-underline"
style={brandFont}
>
ktx
</Link>
<a
href="https://www.kaelio.com"
target="_blank"
rel="noreferrer"
className="mt-1 whitespace-nowrap text-[13px] font-medium text-fd-muted-foreground/80 tracking-tight no-underline transition-colors hover:text-fd-foreground"
style={brandFont}
>
by Kaelio
</a>
</div>
<div className="flex items-center gap-3.5 group">
<div className="relative flex items-center justify-center transition-transform duration-300 ease-out group-hover:rotate-[-4deg]">
<img
src="/ktx/brand/ktx-mascot.svg"
alt=""
aria-hidden="true"
className="h-20 w-20 object-contain block dark:hidden"
/>
<img
src="/ktx/brand/ktx-mascot-dark.svg"
alt=""
aria-hidden="true"
className="h-20 w-20 object-contain hidden dark:block"
/>
</div>
<div className="flex flex-col items-start leading-none">
<span
className="text-[19px] font-medium text-fd-muted-foreground/80 tracking-tight border-l border-fd-border pl-3 ml-1"
style={brandFont}
className="text-[42px] font-semibold text-fd-foreground tracking-tight"
style={{ fontFamily: "var(--font-display), var(--font-sans), sans-serif" }}
>
Docs
ktx
</span>
<span
className="mt-1 whitespace-nowrap text-[13px] font-medium text-fd-muted-foreground/80 tracking-tight"
style={{ fontFamily: "var(--font-display), var(--font-sans), sans-serif" }}
>
by Kaelio
</span>
</div>
<span
className="text-[19px] font-medium text-fd-muted-foreground/80 tracking-tight border-l border-fd-border pl-3 ml-1"
style={{ fontFamily: "var(--font-display), var(--font-sans), sans-serif" }}
>
Docs
</span>
</div>
);
}

View file

@ -68,14 +68,14 @@ const EDGE_STROKE = "#94a3b8";
const sourceData: SourceNodeData[] = [
{
title: "Databases",
body: "Schemas, keys, row counts, query history.",
items: ["PostgreSQL", "Snowflake", "BigQuery", "Athena", "MongoDB", "& more"],
body: "Schemas, columns, keys, row counts, and query history.",
items: ["PostgreSQL", "Snowflake", "BigQuery", "SQLite"],
accent: "#3b82f6",
},
{
title: "BI tools",
body: "Dashboards, questions, explores, usage, and trusted examples.",
items: ["Metabase", "Looker", "Sigma"],
items: ["Metabase", "Looker"],
accent: "#f97316",
},
{
@ -87,7 +87,7 @@ const sourceData: SourceNodeData[] = [
{
title: "Docs and notes",
body: "Policies, caveats, team definitions, and analyst context.",
items: ["Notion", "Google Drive", "Any text"],
items: ["Notion", "Any text"],
accent: "#10b981",
},
];

View file

@ -1,576 +0,0 @@
"use client";
import {
type Edge,
type EdgeProps,
getSmoothStepPath,
Handle,
MarkerType,
type Node,
type NodeProps,
Position,
} from "@xyflow/react";
import { FlowCanvas } from "./flow-canvas";
type AgentNodeData = {
title: string;
items: string[];
};
type HubNodeData = {
title: string;
badge: string;
rows: string[];
};
type TargetNodeData = {
accent: string;
title: string;
body: string;
rows: { text: string; color?: string; mono?: boolean }[];
badge?: string;
};
type AgentNode = Node<AgentNodeData, "agent">;
type HubNode = Node<HubNodeData, "hub">;
type TargetNode = Node<TargetNodeData, "target">;
type FlowNode = AgentNode | HubNode | TargetNode;
const AGENT_W = 252;
const AGENT_H = 96;
const HUB_W = 306;
const HUB_H = 190;
const TARGET_W = 268;
const TARGET_H = 148;
const CENTER_X = 470;
const ROW_AGENT_Y = 0;
const ROW_HUB_Y = 196;
const ROW_TARGET_Y = 488;
const AGENT_X = CENTER_X - AGENT_W / 2;
const HUB_X = CENTER_X - HUB_W / 2;
const TARGET_GAP_X = 38;
const TARGETS_TOTAL = TARGET_W * 2 + TARGET_GAP_X;
const TARGETS_START_X = CENTER_X - TARGETS_TOTAL / 2;
const CONTEXT_X = TARGETS_START_X;
const WAREHOUSE_X = TARGETS_START_X + TARGET_W + TARGET_GAP_X;
const EDGE_STROKE = "#94a3b8";
const CYCLE_STROKE = "#0e7490";
const EMERALD = "#059669";
const TEAL = "#0e7490";
const nodes: FlowNode[] = [
{
id: "agent",
type: "agent",
position: { x: AGENT_X, y: ROW_AGENT_Y },
data: {
title: "Your agent",
items: ["Claude Code", "Cursor", "Codex"],
},
draggable: false,
selectable: false,
},
{
id: "hub",
type: "hub",
position: { x: HUB_X, y: ROW_HUB_Y },
data: {
title: "ktx",
badge: "MCP + CLI",
rows: [
"Search wiki + semantic layer",
"Return approved metrics",
"Compile metrics → SQL",
],
},
draggable: false,
selectable: false,
},
{
id: "context",
type: "target",
position: { x: CONTEXT_X, y: ROW_TARGET_Y },
data: {
accent: TEAL,
title: "Context layer",
body: "Approved definitions agents search before they answer.",
rows: [
{ text: "wiki/*.md", color: EMERALD, mono: true },
{ text: "semantic-layer/*.yaml", color: TEAL, mono: true },
],
},
draggable: false,
selectable: false,
},
{
id: "warehouse",
type: "target",
position: { x: WAREHOUSE_X, y: ROW_TARGET_Y },
data: {
accent: "#334155",
title: "Database",
badge: "read-only",
body: "Runs the compiled SQL. ktx never writes to it.",
rows: [],
},
draggable: false,
selectable: false,
},
];
const labelBg = {
labelBgPadding: [6, 3] as [number, number],
labelBgBorderRadius: 4,
labelStyle: {
fontSize: 13,
fontWeight: 600,
fill: "var(--color-fd-muted-foreground)",
},
labelBgStyle: {
fill: "var(--color-fd-background)",
stroke: "var(--color-fd-border)",
strokeWidth: 1,
},
};
const requestMarker = {
type: MarkerType.ArrowClosed,
color: EDGE_STROKE,
width: 16,
height: 16,
};
const flowEdges: Edge[] = [
{
id: "e-ask",
source: "agent",
sourceHandle: "ask",
target: "hub",
targetHandle: "ask",
type: "straight",
label: "ask",
...labelBg,
style: { stroke: EDGE_STROKE, strokeWidth: 1.5 },
markerEnd: requestMarker,
},
{
id: "e-answer",
source: "hub",
sourceHandle: "answer",
target: "agent",
targetHandle: "answer",
type: "straight",
label: "answer",
...labelBg,
style: { stroke: EDGE_STROKE, strokeWidth: 1.5 },
markerEnd: requestMarker,
},
{
id: "e-search",
source: "hub",
sourceHandle: "to-context",
target: "context",
targetHandle: "in",
type: "smoothstep",
label: "search + read",
...labelBg,
style: { stroke: CYCLE_STROKE, strokeWidth: 1.5 },
markerStart: { type: MarkerType.ArrowClosed, color: CYCLE_STROKE, width: 14, height: 14 },
markerEnd: { type: MarkerType.ArrowClosed, color: CYCLE_STROKE, width: 14, height: 14 },
},
{
id: "e-readonly",
source: "hub",
sourceHandle: "to-warehouse",
target: "warehouse",
targetHandle: "in",
type: "smoothstep",
label: "read-only",
...labelBg,
style: { stroke: CYCLE_STROKE, strokeWidth: 1.5 },
markerStart: { type: MarkerType.ArrowClosed, color: CYCLE_STROKE, width: 14, height: 14 },
markerEnd: { type: MarkerType.ArrowClosed, color: CYCLE_STROKE, width: 14, height: 14 },
},
];
function AgentNodeView({ data }: NodeProps<AgentNode>) {
return (
<div
style={{ width: AGENT_W, height: AGENT_H }}
className="flex flex-col justify-center rounded-md border border-fd-border bg-fd-card px-3.5 py-2.5 shadow-sm"
>
<Handle
id="ask"
type="source"
position={Position.Bottom}
className="!opacity-0"
style={{ left: "35%" }}
/>
<Handle
id="answer"
type="target"
position={Position.Bottom}
className="!opacity-0"
style={{ left: "65%" }}
/>
<div className="flex items-center gap-2.5">
<span className="flex h-8 w-8 flex-none items-center justify-center rounded-full bg-fd-primary/15 text-fd-primary">
<svg
xmlns="http://www.w3.org/2000/svg"
width="18"
height="18"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="1.75"
strokeLinecap="round"
strokeLinejoin="round"
aria-hidden="true"
>
<rect x="3" y="6" width="18" height="12" rx="3" />
<circle cx="9" cy="12" r="1.25" fill="currentColor" stroke="none" />
<circle cx="15" cy="12" r="1.25" fill="currentColor" stroke="none" />
<path d="M12 3v3" />
</svg>
</span>
<p className="text-[17px] font-semibold leading-6 text-fd-foreground">
{data.title}
</p>
</div>
<div className="mt-2 flex flex-wrap gap-1.5">
{data.items.map((item) => (
<span
key={item}
className="rounded border border-fd-border bg-fd-background px-1.5 py-0.5 text-[12px] leading-5 text-fd-muted-foreground"
>
{item}
</span>
))}
</div>
</div>
);
}
function HubNodeView({ data }: NodeProps<HubNode>) {
return (
<div
style={{ width: HUB_W, height: HUB_H }}
className="relative flex flex-col rounded-md border border-cyan-200/20 bg-[#0f1f23] px-4 py-3.5 text-white shadow-sm dark:bg-[#0b181b]"
>
<Handle
id="ask"
type="target"
position={Position.Top}
className="!opacity-0"
style={{ left: "37.5%" }}
/>
<Handle
id="answer"
type="source"
position={Position.Top}
className="!opacity-0"
style={{ left: "62.5%" }}
/>
<Handle
id="to-context"
type="source"
position={Position.Bottom}
className="!opacity-0"
style={{ left: "44%" }}
/>
<Handle
id="to-warehouse"
type="source"
position={Position.Bottom}
className="!opacity-0"
style={{ left: "56%" }}
/>
<div className="flex items-center gap-2.5">
<span className="flex h-7 w-7 flex-none items-center justify-center rounded-md bg-cyan-300/95 font-mono text-sm font-bold text-[#0b1c20]">
k
</span>
<span className="text-[19px] font-bold leading-6 text-white">
{data.title}
</span>
<span className="ml-1 rounded border border-cyan-200/30 bg-white/5 px-1.5 py-0.5 font-mono text-[11px] leading-5 text-cyan-100/85">
{data.badge}
</span>
</div>
<div className="mt-3 flex flex-1 flex-col justify-center gap-2">
{data.rows.map((row) => (
<div key={row} className="flex items-center gap-2.5">
<span className="h-1.5 w-1.5 flex-none rounded-full bg-cyan-300/95" />
<span className="text-[14px] font-medium leading-5 text-cyan-50/90">
{row}
</span>
</div>
))}
</div>
</div>
);
}
function TargetNodeView({ data }: NodeProps<TargetNode>) {
return (
<div
style={{
width: TARGET_W,
height: TARGET_H,
borderTop: `3px solid ${data.accent}`,
}}
className="overflow-hidden rounded-md border border-fd-border bg-fd-card px-3.5 py-3 shadow-sm"
>
<Handle id="in" type="target" position={Position.Top} className="!opacity-0" />
<div className="flex items-center gap-2">
<p className="text-[17px] font-semibold leading-6 text-fd-foreground">
{data.title}
</p>
{data.badge ? (
<span
className="rounded-full px-1.5 py-0.5 text-[11px] font-semibold leading-5"
style={{
color: data.accent,
background: "color-mix(in oklch, var(--color-fd-card) 86%, #64748b)",
}}
>
{data.badge}
</span>
) : null}
</div>
{data.rows.length > 0 ? (
<div className="mt-1 flex flex-col gap-0.5">
{data.rows.map((row) => (
<span
key={row.text}
className={
row.mono
? "font-mono text-[13px] font-semibold tracking-tight"
: "text-[12px] leading-4 text-fd-muted-foreground"
}
style={row.color ? { color: row.color } : undefined}
>
{row.text}
</span>
))}
</div>
) : null}
<p className="mt-1.5 line-clamp-2 text-[13px] leading-[18px] text-fd-muted-foreground">
{data.body}
</p>
</div>
);
}
/* ------------------------------- Particles ------------------------------- */
const PARTICLE_SPEED_PX_PER_SEC = 150;
const PARTICLE_MIN_DURATION_SEC = 5;
type Leg = {
sx: number;
sy: number;
sPos: Position;
tx: number;
ty: number;
tPos: Position;
};
const AGENT_ASK_X = AGENT_X + AGENT_W * 0.35;
const AGENT_ANSWER_X = AGENT_X + AGENT_W * 0.65;
const AGENT_BOTTOM_Y = ROW_AGENT_Y + AGENT_H;
const HUB_ASK_X = HUB_X + HUB_W * 0.375;
const HUB_ANSWER_X = HUB_X + HUB_W * 0.625;
const HUB_TO_CONTEXT_X = HUB_X + HUB_W * 0.44;
const HUB_TO_WAREHOUSE_X = HUB_X + HUB_W * 0.56;
const HUB_BOTTOM_Y = ROW_HUB_Y + HUB_H;
const CONTEXT_TOP_X = CONTEXT_X + TARGET_W / 2;
const WAREHOUSE_TOP_X = WAREHOUSE_X + TARGET_W / 2;
function buildCyclePath(spokeX: number, targetX: number): {
d: string;
length: number;
} {
const legs: Leg[] = [
// agent → hub (ask, down)
{ sx: AGENT_ASK_X, sy: AGENT_BOTTOM_Y, sPos: Position.Bottom, tx: HUB_ASK_X, ty: ROW_HUB_Y, tPos: Position.Top },
// through the hub to its spoke handle (down, drawn behind the hub)
{ sx: HUB_ASK_X, sy: ROW_HUB_Y, sPos: Position.Bottom, tx: spokeX, ty: HUB_BOTTOM_Y, tPos: Position.Top },
// hub → target (down)
{ sx: spokeX, sy: HUB_BOTTOM_Y, sPos: Position.Bottom, tx: targetX, ty: ROW_TARGET_Y, tPos: Position.Top },
// target → hub (up)
{ sx: targetX, sy: ROW_TARGET_Y, sPos: Position.Top, tx: spokeX, ty: HUB_BOTTOM_Y, tPos: Position.Bottom },
// through the hub to its answer handle (up, drawn behind the hub)
{ sx: spokeX, sy: HUB_BOTTOM_Y, sPos: Position.Top, tx: HUB_ANSWER_X, ty: ROW_HUB_Y, tPos: Position.Bottom },
// hub → agent (answer, up)
{ sx: HUB_ANSWER_X, sy: ROW_HUB_Y, sPos: Position.Top, tx: AGENT_ANSWER_X, ty: AGENT_BOTTOM_Y, tPos: Position.Bottom },
];
const segments = legs.map((leg) => {
const [segment] = getSmoothStepPath({
sourceX: leg.sx,
sourceY: leg.sy,
sourcePosition: leg.sPos,
targetX: leg.tx,
targetY: leg.ty,
targetPosition: leg.tPos,
});
return segment;
});
let d = segments[0];
for (let i = 1; i < segments.length; i += 1) {
d += ` ${segments[i].replace(/^M/, "L")}`;
}
const length = legs.reduce(
(sum, leg) => sum + Math.abs(leg.tx - leg.sx) + Math.abs(leg.ty - leg.sy),
0,
);
return { d, length };
}
type ParticleEdgeData = {
d: string;
duration: number;
beginOffset: number;
color: string;
};
type ParticleEdge = Edge<ParticleEdgeData, "particle">;
function ParticleEdgeView({ id, data }: EdgeProps<ParticleEdge>) {
if (!data) return null;
const pathId = `runtime-particle-path-${id}`;
return (
<>
<path id={pathId} d={data.d} fill="none" stroke="none" pointerEvents="none" />
<g className="runtime-particle" style={{ color: data.color }}>
<circle r={7.5} fill="currentColor" opacity={0.16} />
<circle r={3.75} fill="currentColor" opacity={0.32} />
<circle r={2.1} fill="currentColor" />
<animateMotion
dur={`${data.duration.toFixed(2)}s`}
begin={`-${data.beginOffset.toFixed(2)}s`}
repeatCount="indefinite"
>
<mpath href={`#${pathId}`} />
</animateMotion>
</g>
</>
);
}
function makeCycleEdge(
id: string,
source: string,
spokeX: number,
targetX: number,
beginFraction: number,
): ParticleEdge {
const { d, length } = buildCyclePath(spokeX, targetX);
const duration = Math.max(
PARTICLE_MIN_DURATION_SEC,
length / PARTICLE_SPEED_PX_PER_SEC,
);
return {
id,
source,
target: source,
type: "particle",
data: { d, duration, beginOffset: duration * beginFraction, color: CYCLE_STROKE },
};
}
const particleEdges: ParticleEdge[] = [
makeCycleEdge("p-context", "context", HUB_TO_CONTEXT_X, CONTEXT_TOP_X, 0),
makeCycleEdge("p-warehouse", "warehouse", HUB_TO_WAREHOUSE_X, WAREHOUSE_TOP_X, 0.5),
];
const nodeTypes = {
agent: AgentNodeView,
hub: HubNodeView,
target: TargetNodeView,
};
const edgeTypes = {
particle: ParticleEdgeView,
};
const edges = [...flowEdges, ...particleEdges];
export function ProductRuntime() {
return (
<section
className="not-prose my-12 w-full max-w-full min-w-0 space-y-5"
aria-labelledby="runtime-title"
>
<div className="max-w-3xl">
<h2
id="runtime-title"
className="text-xl font-semibold tracking-normal text-fd-foreground sm:text-2xl"
style={{ fontFamily: "var(--font-display)" }}
>
How serving works
</h2>
<p className="mt-3 text-sm leading-6 text-fd-muted-foreground">
At runtime, agents reach ktx through MCP. ktx searches the context
layer, returns approved metrics, and compiles them into read-only SQL
the warehouse runs.
</p>
</div>
<article
className="max-w-full min-w-0 overflow-hidden rounded-lg border border-fd-border bg-fd-card shadow-sm"
aria-label="ktx serving flow from an agent request to a governed answer"
>
<div className="border-b border-fd-border bg-fd-muted/35 px-5 py-4">
<p className="text-xs font-semibold uppercase tracking-wide text-fd-primary">
Serving flow
</p>
<h3
className="mt-1 text-base font-semibold tracking-normal text-fd-foreground sm:text-lg"
style={{ fontFamily: "var(--font-display)" }}
>
From an agent request to a governed answer
</h3>
<p className="mt-2 max-w-3xl text-xs leading-5 text-fd-muted-foreground">
The agent asks in plain language. ktx is the only thing that touches
the context layer and the warehouse, and every database connection
is read-only.
</p>
</div>
<FlowCanvas
nodes={nodes}
edges={edges}
nodeTypes={nodeTypes}
edgeTypes={edgeTypes}
canvasStyle={{
height: "min(620px, 98vw)",
minHeight: 430,
}}
className="runtime-canvas"
fitViewOptions={{ padding: 0.06 }}
ariaLabel="ktx serving flow diagram"
/>
</article>
<style>{`
.runtime-canvas .runtime-particle {
pointer-events: none;
filter: drop-shadow(0 0 6px currentColor);
}
@media (prefers-reduced-motion: reduce) {
.runtime-canvas .runtime-particle {
display: none;
}
}
`}</style>
</section>
);
}

View file

@ -1,157 +0,0 @@
"use client";
import {
useEffect,
useRef,
useState,
type ComponentProps,
type KeyboardEvent,
type SVGProps,
} from "react";
import { useTheme } from "fumadocs-ui/provider/base";
/**
* Three-icon theme switcher (light / system / dark) rendered as a radio group
* each icon selects its own theme, unlike fumadocs' default "light-dark"
* switcher, which is a single blind toggle that flips on any click. Reads
* `theme`, not `resolvedTheme`, so the "system" option can show as selected
* (resolvedTheme collapses system to light/dark). Dropped into the sidebar
* footer pill via `slots.themeSwitch`, so fumadocs passes the container
* className (left divider, `ms-auto`, rounded inner buttons); we merge it onto
* our own base.
*
* Icons are inlined (the project doesn't depend on `lucide-react` directly);
* `useTheme` is re-exported by fumadocs so we avoid a bare `next-themes` import.
*/
function SunIcon(props: SVGProps<SVGSVGElement>) {
return (
<svg
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth={2}
strokeLinecap="round"
strokeLinejoin="round"
aria-hidden="true"
{...props}
>
<circle cx="12" cy="12" r="4" />
<path d="M12 2v2" />
<path d="M12 20v2" />
<path d="m4.93 4.93 1.41 1.41" />
<path d="m17.66 17.66 1.41 1.41" />
<path d="M2 12h2" />
<path d="M20 12h2" />
<path d="m6.34 17.66-1.41 1.41" />
<path d="m19.07 4.93-1.41 1.41" />
</svg>
);
}
function MonitorIcon(props: SVGProps<SVGSVGElement>) {
return (
<svg
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth={2}
strokeLinecap="round"
strokeLinejoin="round"
aria-hidden="true"
{...props}
>
<rect x="3" y="4" width="18" height="12" rx="2" />
<path d="M8 20h8" />
<path d="M12 16v4" />
</svg>
);
}
function MoonIcon(props: SVGProps<SVGSVGElement>) {
return (
<svg
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth={2}
strokeLinecap="round"
strokeLinejoin="round"
aria-hidden="true"
{...props}
>
<path d="M12 3a6 6 0 0 0 9 9 9 9 0 1 1-9-9Z" />
</svg>
);
}
const OPTIONS = [
["light", SunIcon],
["system", MonitorIcon],
["dark", MoonIcon],
] as const;
function cx(...classes: (string | false | undefined)[]): string {
return classes.filter(Boolean).join(" ");
}
export function ThemeToggle({ className, ...props }: ComponentProps<"div">) {
const { theme, setTheme } = useTheme();
const [mounted, setMounted] = useState(false);
useEffect(() => setMounted(true), []);
const active = mounted ? theme : null;
const buttonsRef = useRef<(HTMLButtonElement | null)[]>([]);
// Pre-mount nothing is selected, so keep the first control tabbable.
const selectedIndex = OPTIONS.findIndex(([key]) => key === active);
const rovingIndex = selectedIndex === -1 ? 0 : selectedIndex;
// Radio-group keyboard model: arrows move focus and pick that theme.
function onKeyDown(event: KeyboardEvent<HTMLButtonElement>, index: number) {
const delta =
event.key === "ArrowRight" || event.key === "ArrowDown"
? 1
: event.key === "ArrowLeft" || event.key === "ArrowUp"
? -1
: 0;
if (delta === 0) return;
event.preventDefault();
const next = (index + delta + OPTIONS.length) % OPTIONS.length;
setTheme(OPTIONS[next][0]);
buttonsRef.current[next]?.focus();
}
return (
<div
className={cx("inline-flex items-center overflow-hidden border", className)}
data-theme-toggle=""
role="radiogroup"
aria-label="Theme"
{...props}
>
{OPTIONS.map(([key, Icon], index) => (
<button
key={key}
ref={(el) => {
buttonsRef.current[index] = el;
}}
type="button"
role="radio"
aria-label={key}
aria-checked={active === key}
tabIndex={index === rovingIndex ? 0 : -1}
onClick={() => setTheme(key)}
onKeyDown={(event) => onKeyDown(event, index)}
className={cx(
"size-6.5 p-1.5 transition-colors",
active === key
? "bg-fd-accent text-fd-accent-foreground"
: "text-fd-muted-foreground hover:text-fd-accent-foreground",
)}
>
<Icon className="size-full" />
</button>
))}
</div>
);
}

View file

@ -0,0 +1,40 @@
---
title: Agent Instructions
description: Suggested instructions for coding assistants that need to read and cite ktx docs.
---
Use these instructions when a coding assistant needs to answer questions from the **ktx** documentation.
```text
When answering ktx docs questions:
1. Start with https://docs.kaelio.com/ktx/llms.txt.
2. Fetch the smallest relevant Markdown page from the index.
3. Prefer /docs/<path>.md over rendered HTML.
4. Use https://docs.kaelio.com/ktx/llms-full.txt only when the task needs broad docs context.
5. Quote commands exactly from docs pages.
6. If docs and local repository behavior disagree, say what differs and prefer local verified output for code changes.
```
## What this is for
This page is for documentation consumption only:
- answering questions about **ktx**
- finding the right docs page
- citing setup or CLI guidance
- helping an assistant avoid stale or invented commands
It does not describe local tool configuration.
## Minimal project prompt
```text
You are helping with ktx. Read https://docs.kaelio.com/ktx/llms.txt first, then fetch only the Markdown pages needed for the task. Do not scrape the rendered docs site when a .md route exists.
```
## Repository prompt
```text
Before editing ktx docs, read /llms.txt and the affected .md docs pages. Keep AI Resources focused on docs consumption. After editing, verify /llms.txt, /llms-full.txt, and any changed .md routes.
```

View file

@ -0,0 +1,54 @@
---
title: Agent Quickstart
description: A task-first route for coding agents that need to understand ktx docs.
---
This page is for coding assistants reading or citing the **ktx** docs. It is intentionally limited to documentation lookup, docs navigation, and safe command discovery.
For Markdown endpoints, use [Markdown Access](/docs/ai-resources/markdown-access).
For reusable task prompts, use [Prompt Recipes](/docs/ai-resources/prompt-recipes).
To install **ktx** into an agent client, use [Agent Clients](/docs/integrations/agent-clients).
## First read
Agents should start with the smallest source that answers the task:
1. [`/llms.txt`](/llms.txt) - discover the docs and preferred entry points.
2. The relevant per-page Markdown URL, for example `/docs/getting-started/quickstart.md`.
3. [`/llms-full.txt`](/llms-full.txt) - use only when the task needs broad context across many pages.
## Task router
| User asks the agent to explain... | Read first | Then read |
|------------------------------------|------------|-----------|
| What **ktx** does | [Introduction](/docs/getting-started/introduction) | [The Context Layer](/docs/concepts/the-context-layer) |
| How to start from a checkout | [Quickstart](/docs/getting-started/quickstart) | [ktx setup](/docs/cli-reference/ktx-setup) |
| How to check project readiness | [ktx status](/docs/cli-reference/ktx-status) | [Quickstart](/docs/getting-started/quickstart) |
| How context gets built | [Building Context](/docs/guides/building-context) | [ktx ingest](/docs/cli-reference/ktx-ingest) |
| How semantic YAML works | [Writing Context](/docs/guides/writing-context) | [ktx sl](/docs/cli-reference/ktx-sl) |
| How machine-readable CLI output is shaped | [ktx sl](/docs/cli-reference/ktx-sl) | [ktx wiki](/docs/cli-reference/ktx-wiki) |
## Operating workflow
Use this workflow when the user asks an assistant to answer a **ktx** docs question:
1. Read [`/llms.txt`](/llms.txt).
2. Pick the smallest relevant `.md` page.
3. Use [`/llms-full.txt`](/llms-full.txt) only if the answer needs multiple sections of the docs.
4. Quote commands exactly from the docs page.
5. If a command affects a local project, ask the user before assuming credentials or live services are available.
## Docs lookup from a shell
```bash
curl https://docs.kaelio.com/ktx/llms.txt
curl https://docs.kaelio.com/ktx/docs/getting-started/quickstart.md
```
## Guardrails
- Do not invent CLI flags. Fetch the relevant CLI reference page.
- Do not scrape rendered HTML when a `.md` route exists.
- Do not assume docs lookup requires agent-client configuration.
- Do not include credentials or secrets in prompts, URLs, or copied docs snippets.
- When docs and local CLI behavior disagree, prefer the local CLI output and mention the mismatch.

View file

@ -0,0 +1,76 @@
---
title: Markdown Access
description: Fetch ktx docs as llms.txt, llms-full.txt, or per-page Markdown.
---
**ktx** docs are available as plain Markdown so assistants do not need to parse the rendered HTML site.
## Index
Fetch the curated index:
```text
https://docs.kaelio.com/ktx/llms.txt
```
Use this file to discover high-value pages, task-specific entry points, and Markdown URLs.
## Full corpus
Fetch the complete docs corpus:
```text
https://docs.kaelio.com/ktx/llms-full.txt
```
Use this when an assistant needs broad context across setup, concepts, CLI reference, integrations, and troubleshooting. Prefer the smaller per-page Markdown route for narrow tasks.
## Per-page Markdown
Every docs page has a Markdown route:
```text
https://docs.kaelio.com/ktx/docs/getting-started/quickstart.md
https://docs.kaelio.com/ktx/docs/cli-reference/ktx-sl.md
https://docs.kaelio.com/ktx/docs/cli-reference/ktx-wiki.md
https://docs.kaelio.com/ktx/docs/guides/building-context.md
```
Requests that ask for Markdown can also use the normal docs URL with `Accept: text/markdown`:
```bash
curl -H "Accept: text/markdown" https://docs.kaelio.com/ktx/docs/getting-started/quickstart
```
## Recommended retrieval order
1. Fetch `/llms.txt`.
2. Select one or two relevant page Markdown URLs.
3. Fetch `/llms-full.txt` only when page-level docs are not enough.
## Output contract
Markdown responses are designed for agent consumption:
- Frontmatter is removed.
- Each page includes a title, description, canonical URL, and Markdown URL.
- Code blocks stay as code blocks.
- Tables stay as Markdown tables.
- Missing docs pages return a plain-text `404` instead of silently falling back to HTML.
## Page actions
Rendered docs pages include page-level actions near the title:
- **Copy MD** copies the generated Markdown for the current page.
- **View MD** opens the generated Markdown route.
- **Copy MDX** copies the source MDX for the current page.
## Common mistakes
| Mistake | Better path |
|---------|-------------|
| Scraping the HTML page for a docs answer | Fetch the `.md` route instead |
| Loading `/llms-full.txt` for a single CLI flag lookup | Fetch the relevant CLI reference page |
| Treating `/llms.txt` as complete documentation | Use it as an index, then fetch linked pages |
| Copying rendered text by hand | Use **Copy MD** or **Copy MDX** from the page actions |

View file

@ -0,0 +1,10 @@
{
"title": "AI Resources",
"defaultOpen": true,
"pages": [
"agent-quickstart",
"markdown-access",
"agent-instructions",
"prompt-recipes"
]
}

View file

@ -0,0 +1,55 @@
---
title: Prompt Recipes
description: Copyable prompts for common ktx agent workflows.
---
Use these prompts when asking a coding assistant to work with **ktx**. Replace project names, connection ids, and business terms with your own values.
## Learn the docs
```text
Read https://docs.kaelio.com/ktx/llms.txt first. Then fetch only the ktx Markdown pages needed for this task. Do not scrape rendered HTML unless no Markdown route exists.
```
## Set up a project
```text
Run npx skills add Kaelio/ktx --skill ktx and use the ktx skill to install
and configure ktx in this project.
```
## Find a command
```text
Find the correct ktx command for this task: <task>. Start with /llms.txt, then fetch the smallest relevant CLI reference .md page. Quote the exact command and flags from the docs.
```
## Explain setup
```text
Explain how to set up ktx for this repo. Read /docs/getting-started/quickstart.md and the relevant CLI reference pages. Summarize prerequisites, commands, generated files, and any credentials the user must provide manually.
```
## Compare concepts
```text
Explain the difference between these ktx concepts: <concepts>. Start from /llms.txt, fetch the relevant concept and guide pages as Markdown, and answer with links to the source pages.
```
## Review semantic changes
```text
Review the ktx semantic-layer and knowledge changes in this branch. Check that measures have clear definitions, joins use valid keys, hidden/internal columns are not exposed to agents, and validation passes. List concrete file and line issues first.
```
## Copy exact docs source
```text
Open the relevant ktx docs page and use the page action to copy the generated Markdown or source MDX. Preserve code fences and tables exactly.
```
## Update docs
```text
Update the ktx docs for agent readability. Keep AI Resources focused on docs consumption. After editing, verify /llms.txt, /llms-full.txt, and the affected .md routes.
```

View file

@ -48,11 +48,6 @@ directory. Use it from any directory to generate editor or agent schema files.
| `stop` | Stop the **ktx** daemon |
| `status` | Show managed Python runtime status and readiness checks |
`install` is self-contained: **ktx** downloads its own pinned, checksum-verified
`uv` build under the runtime root and uses it to provision Python and the
runtime wheel. Nothing needs to be installed on `PATH` first; the host only
needs network access to `github.com` during the first install.
## `admin runtime` Options
| Flag | Description | Default |

View file

@ -1,86 +0,0 @@
---
title: "ktx completion"
description: "Print a shell completion script for tab completion."
---
Print a shell completion script for **ktx**. Once installed, pressing <kbd>Tab</kbd>
completes commands, subcommands, and flags, and - inside a **ktx** project - the
names of things that already exist: semantic-layer source names for
`ktx sl read` and `ktx sl validate`, wiki page keys for `ktx wiki read`, and
configured connection ids for `ktx connection test`, `ktx ingest`, and
`ktx sql`. This saves you from remembering exact source, page, or connection
names.
## Command signature
```bash
ktx completion <shell>
```
`<shell>` must be `zsh` or `bash`. The command writes the script to stdout; it
does not modify any files. Enable completion by evaluating the script in your
shell startup file.
## Installation
Add the matching line to your shell startup file, then restart your shell (or
`source` the file). `ktx` must be on your `PATH`.
```bash
# zsh — add to ~/.zshrc
eval "$(ktx completion zsh)"
```
```bash
# bash — add to ~/.bashrc
eval "$(ktx completion bash)"
```
To try it for the current session only, run the same `eval` line directly in
your terminal.
## What gets completed
| Position | Completions |
|----------|-------------|
| `ktx <Tab>` | Top-level commands (`setup`, `sl`, `wiki`, `ingest`, …) |
| `ktx sl <Tab>` | The `read` / `validate` / `query` subcommands |
| `ktx sl read <Tab>` | Existing semantic-layer source names |
| `ktx sl validate <Tab>` | Existing semantic-layer source names |
| `ktx wiki <Tab>` | The `read` subcommand |
| `ktx wiki read <Tab>` | Existing wiki page keys |
| `ktx connection test <Tab>` | Configured connection ids |
| `ktx ingest <Tab>` | Configured connection ids |
| `ktx sql --connection <Tab>` | Configured connection ids |
| `ktx completion <Tab>` | `zsh` or `bash` |
| `ktx <command> --<Tab>` | The command's flags and inherited global flags |
| `ktx sl --output <Tab>` | An option's allowed values (here `pretty`, `plain`, `json`) |
| `ktx sl --connection-id <Tab>` | Configured connection ids |
Source names, wiki page keys, and connection ids are read from the **ktx**
project resolved from your current directory (or `--project-dir` /
`KTX_PROJECT_DIR`). Outside a **ktx** project, completion still suggests
commands and flags but no project entities. Bare `ktx sl <Tab>` and
`ktx wiki <Tab>` complete subcommands instead of entity names because their
positional arguments are free-text search queries.
## Examples
```bash
# Print the zsh completion script
ktx completion zsh
# Print the bash completion script
ktx completion bash
# Install for zsh
echo 'eval "$(ktx completion zsh)"' >> ~/.zshrc
```
## Common errors
| Error | Cause | Recovery |
|-------|-------|----------|
| `error: command-argument value '<name>' is invalid for argument 'shell'. Allowed choices are zsh, bash.` | A shell other than `zsh` or `bash` was requested | Re-run with `ktx completion zsh` or `ktx completion bash` |
| Tab completion does nothing | The script was not evaluated, or `ktx` is not on `PATH` | Confirm the `eval` line is in your startup file, restart the shell, and verify `ktx --version` runs |
| Source, page, or connection names are missing | The current directory is not inside a **ktx** project | Run from the project directory, or pass `--project-dir`, or set `KTX_PROJECT_DIR` |

View file

@ -104,6 +104,6 @@ configured connection and exit non-zero if any probe fails.
| Error | Cause | Recovery |
|-------|-------|----------|
| No connections configured | The project has no entries under `connections` | Run `ktx setup` and add a database or context-source connection |
| Connection test fails | Credentials, network access, database, warehouse, or schema is invalid | Use the setup recovery menu to retry or re-enter details; if it still fails, verify the same URL with the database's native client |
| Mapping validation fails during setup | BI database mappings do not point at valid warehouse connections | Use the setup recovery menu to retry validation or re-enter mapping selections; rerun `ktx setup` if you already exited |
| Connection test fails | Credentials, network access, database, warehouse, or schema is invalid | Verify the same URL with the database's native client, then rerun `ktx setup` and reconfigure the connection |
| Mapping validation fails during setup | BI database mappings do not point at valid warehouse connections | Rerun `ktx setup` and update the context-source mapping selections |
| Notion page picker cannot run | The terminal is non-interactive or Notion discovery failed | Rerun interactive `ktx setup`, or use non-interactive setup flags with explicit root page ids |

View file

@ -5,11 +5,9 @@ description: "Build or refresh ktx context, or capture text into ktx memory."
`ktx ingest` builds or refreshes **ktx** context from configured connections, and
can also capture free-form text into **ktx** memory. Database connections build
enriched context — schema plus AI-generated descriptions, embeddings, and
relationship evidence — and require a configured model and embeddings.
Context-source connections ingest metadata from tools such as dbt, Looker,
Metabase, MetricFlow, LookML, Notion, and Sigma. Pass `--text` or `--file` to capture
inline text or text files into memory instead.
schema context. Context-source connections ingest metadata from tools such as
dbt, Looker, Metabase, MetricFlow, LookML, and Notion. Pass `--text` or
`--file` to capture inline text or text files into memory instead.
## Command signature
@ -31,13 +29,13 @@ connection is selected.
| Flag | Description | Default |
|------|-------------|---------|
| `--all` | Ingest all configured connections (same as bare invocation) | `false` |
| `--fast` | Use deterministic fast database ingest | Stored connection default, or `fast` |
| `--deep` | Use deep database ingest with AI-generated descriptions, embeddings, and relationship evidence | Stored connection default, or `fast` |
| `--query-history` | Include database query-history usage patterns | Stored connection default |
| `--no-query-history` | Skip database query-history usage patterns for this run | Stored connection default |
| `--query-history-window-days <days>` | BigQuery/Snowflake query-history lookback window for this run | Stored connection default |
| `--stages <list>` | Comma-separated enrichment stages to (re)run: `descriptions`, `embeddings`, `relationships` | All three |
| `--text <content>` | Capture inline text into **ktx** memory; repeatable | `[]` |
| `--file <path>` | Capture a text file into **ktx** memory; use `-` for stdin; repeatable | `[]` |
| `--verbatim` | Store each `--text`/`--file` document body unchanged as a `GLOBAL` wiki page; the LLM derives metadata only | `false` |
| `--connection-id <connectionId>` | **ktx** connection id to tag captured text/file notes | - |
| `--user-id <id>` | Memory user id for text/file capture attribution | `local-cli` |
| `--fail-fast` | Stop after the first failed text/file item | `false` |
@ -46,12 +44,12 @@ connection is selected.
| `--yes` | Install required managed runtime features without prompting | `false` |
| `--no-input` | Disable interactive terminal input | - |
Database ingest always builds enriched context and requires a configured model
and embeddings (run `ktx setup`); connections without that configuration fail
before any work starts. Query-history flags apply only to database connections
`--fast` and `--deep` are mutually exclusive. Depth flags apply only to
database connections. Query-history flags apply only to database connections
that support query history. The window flag applies to BigQuery and Snowflake;
Postgres reads the current `pg_stat_statements` aggregate data instead of a
time-windowed history table. Query-history ingest runs after the schema scan.
time-windowed history table. Query-history ingest runs after fast ingest and
requires deep ingest readiness.
When more than one connection is selected, database ingest runs first, then
context-source ingest and memory updates run for context-source connections.
@ -65,65 +63,6 @@ use `--no-input` to fail fast with install guidance.
`--text` and `--file` cannot be combined with a positional `connectionId` or
`--all`; pass `--connection-id <id>` instead to tag captured notes.
### Verbatim ingest
By default, captured text is routed through the memory agent, which decides what
to persist and may rewrite, condense, split, or re-title it. For *authoritative*
documents — metric definitions, formula specs, runbooks, compliance text — that
paraphrasing is a defect. Add `--verbatim` to store each `--text`/`--file`
document body **unchanged** as a `GLOBAL` wiki page:
- The stored body is the input document, written by code; the LLM never edits it.
It is used only to derive page metadata (`summary`, `tags`, `sl_refs`), and even
that is skipped for fields the document's own frontmatter already sets.
- The page key is deterministic: a `--file` derives it from the filename, inline
`--text` from the document's leading Markdown heading (inline text without a
heading is rejected — pass it as `--file` instead).
- Ingest is idempotent. Re-running the same document is a safe no-op; a different
body at the same key fails loudly rather than overwriting.
- `--verbatim` works with `llm.provider.backend: none` — the only ingest path that
does. With no backend the `summary` is derived from the heading or first
sentence and `tags`/`sl_refs` are left empty; the full body is still stored.
- Existing frontmatter passes through untouched (including fields **ktx** does not
model, such as `effective_date` or `version`); generated metadata only fills
absent fields. `--connection-id <id>` scopes the page to that connection by
setting its `connections` frontmatter.
### Selecting enrichment stages
Database enrichment runs three stages: `descriptions` (one LLM call per table),
`embeddings` (vectors over the schema and descriptions), and `relationships`
(join detection, optionally LLM-proposed). Each stage is cached on a **per-stage
hash of only its own inputs**, so changing one stage's inputs invalidates only
that stage. Switching the description LLM re-runs only `descriptions`; upgrading
the embeddings model re-runs only `embeddings`; turning on
`scan.relationships.llmProposals` re-runs only `relationships`. The expensive
per-table descriptions are never thrown away because an unrelated setting moved.
`--stages <list>` re-runs a chosen subset on an already-ingested connection. A
named stage is **force-recomputed** (it bypasses the completed-stage cache),
while unselected stages are left exactly as they are on disk:
- `ktx ingest warehouse --stages embeddings` — re-embed on a new model, keeping
descriptions and joins.
- `ktx ingest --all --stages relationships --no-query-history` — backfill joins
across every database after enabling `llmProposals`, without re-paying for
descriptions.
- `ktx ingest warehouse --stages descriptions` — re-run thin descriptions (for
example after raising `KTX_ENRICH_LLM_TIMEOUT_MS`). When nothing the
descriptions depend on changed, the per-table resume record means only the
tables that previously failed are re-sent to the LLM.
Stage names are validated: an unknown or empty name (`--stages foo`, `--stages
descriptions,foo`, `--stages ""`) is a hard parse error. Naming all three
(`--stages descriptions,embeddings,relationships`) forces a full enrichment
recompute, which is **not** the same as omitting the flag (omitting resumes
whatever is already done). After a selective run, **ktx** warns
(`enrichment_stage_stale`) when an unselected stage's inputs no longer match what
it was last built from — for example, re-running `descriptions` flags
`embeddings` as stale until you re-run `--stages embeddings`. The warning is
informational; **ktx** never silently cascades the extra work.
## Examples
```bash
@ -133,16 +72,17 @@ ktx ingest
# Build one database or context-source connection
ktx ingest warehouse
# Force deterministic fast database ingest
ktx ingest warehouse --fast
# Force deep database ingest with AI enrichment
ktx ingest warehouse --deep
# Include query-history usage patterns
ktx ingest warehouse --query-history
ktx ingest warehouse --deep --query-history
# Set the lookback window for BigQuery or Snowflake query history
ktx ingest warehouse --query-history-window-days 30
# Re-embed one connection on a new embeddings model (descriptions/joins untouched)
ktx ingest warehouse --stages embeddings
# Backfill LLM-proposed joins across every database without re-describing
ktx ingest --all --stages relationships --no-query-history
# Build a context-source connection
ktx ingest notion
@ -157,12 +97,6 @@ ktx ingest --file docs/revenue-notes.md --connection-id warehouse
# Capture one stdin item
printf "Refunds are excluded from net revenue." | ktx ingest --file -
# Store an authoritative document verbatim (body preserved exactly)
ktx ingest --file docs/rfm-bucket-definitions.md --verbatim
# Store it verbatim and scope it to one connection
ktx ingest --file docs/haversine-formula.md --verbatim --connection-id warehouse
```
## Output
@ -180,21 +114,6 @@ notion skipped skipped done done
Use `--json` when a script or agent needs the selected plan and per-target
results.
## Final validation pruning
At the end of a context-source ingest, **ktx** validates the composed semantic
layer and wiki before saving it. If the final validation finds dangling
references, **ktx** removes the reference instead of failing accepted work. This
can remove joins that point at missing semantic sources, wiki `refs`, wiki
`sl_refs`, and inline wiki body references. If a generated semantic source is
invalid, **ktx** drops that source from the final save.
The stored ingest report records these changes as `finalGatePrunedReferences`
and `finalGateDroppedSources`. The trace emits `final_gate_reference_pruned`,
`final_gate_source_dropped`, `final_gate_prune_committed`, and
`final_gate_prune_finished` events when pruning runs. If validation still fails
after pruning, the ingest fails and the report keeps the final validation error.
## Inspect context-source ingest traces
Context-source ingest writes persistent JSONL traces for postmortem debugging.
@ -230,55 +149,13 @@ verbosity:
KTX_INGEST_TRACE_LEVEL=trace ktx ingest metabase
```
### Profiling a slow ingest
Each timed phase and work unit records a `durationMs` in the trace, and each
agent loop records its step count and token usage. To see where wall-clock time
went, enable profiling and **ktx** prints a rolled-up breakdown to stderr at the
end of the run. There are two ways to turn it on, and two output formats.
Turn it on per run with the `KTX_PROFILE_INGEST` environment variable, or
persistently with `ingest.profile` in `ktx.yaml` (useful for CI or while
iterating on a slow source):
```bash
KTX_PROFILE_INGEST=1 ktx ingest metabase # human-readable table
KTX_PROFILE_INGEST=json ktx ingest metabase # raw JSON for coding agents
```
```yaml
ingest:
profile: true # human table; use "json" for the machine-readable form
```
Both formats report total wall time, time per phase, and the slowest work units,
splitting each work unit's agent-loop time into model time versus tool-execution
time. The `json` form emits the full structured profile (raw milliseconds and
token counts, stable keys) plus a `summary.headline` one-line diagnosis, so a
coding agent can parse it directly instead of scraping the table. If both the env
var and the config request profiling, `json` wins. Example headline:
```text
Slowest phase: reconciliation (2m 05s, 48% of wall time). 2 work units (1 failed), ~88% model generation vs ~12% tools.
```
Work units run serially by default (`ingest.workUnits.maxConcurrency` is `1`);
raise it in `ktx.yaml` if the profile shows the run is bound by serialized
work-unit agent loops. If the provider reports an LLM rate limit, **ktx** shows
a transient wait message and temporarily reduces effective work-unit concurrency
according to `ingest.rateLimit`.
## Common errors
| Error | Cause | Recovery |
|-------|-------|----------|
| Connection not configured | The connection id is not present in `ktx.yaml` | Add the connection with `ktx setup` or update `ktx.yaml` |
| Enrichment is not configured | Database ingest needs a model, embeddings, and scan-enrichment configuration | Run `ktx setup` to configure a model and embeddings |
| Query history is unsupported | The selected database driver does not support query history | Run ingest without query-history flags |
| Deep readiness is missing | `--deep` or query history needs model, embedding, and scan-enrichment configuration | Run `ktx setup` or rerun with `--fast` |
| Query history is unsupported | The selected database driver does not support query history | Run fast ingest without query-history flags |
| Python runtime is missing | The selected ingest target needs runtime-backed SQL analysis or source parsing | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx admin runtime install` command |
| Context-source options were ignored | Query-history flags were supplied for a context-source connection | Omit database-only flags when ingesting context-source connections |
| Context-source options were ignored | Depth and query-history flags were supplied for a context-source connection | Omit database-only flags when ingesting context-source connections |
| Text ingest stops early | `--fail-fast` was used and one item failed | Fix the failed item or rerun without `--fail-fast` to collect all failures |
| `--verbatim requires --text or --file` | `--verbatim` was passed without a document to store | Add `--text` or `--file`, or drop `--verbatim` |
| Inline verbatim text needs a leading heading | `--text --verbatim` content has no `# Heading` to derive a stable key | Add a leading Markdown heading, or pass the content as `--file <path>` |
| A different page already exists at key | A verbatim re-run targeted an existing key with a different body | Use a distinct document name/key, or remove the existing page first |
| Connection scope conflict | Frontmatter `connections` disagrees with `--connection-id` | Remove one so the intended scope is unambiguous |

View file

@ -68,4 +68,3 @@ hosts and origins for browser clients.
| No **ktx** project found | Current directory has no `ktx.yaml` and `KTX_PROJECT_DIR` is unset | Run from a **ktx** project or pass `--project-dir <path>` |
| Non-loopback host rejected | The server needs token auth before binding beyond localhost | Pass `--token <token>` or set `KTX_MCP_TOKEN` |
| Client cannot connect | Host, port, token, allowed host, or allowed origin does not match the client | Check `ktx mcp status`, then restart with explicit `--host`, `--port`, `--allowed-host`, and `--allowed-origin` values |
| A Python-backed tool reports a runtime install failure | A tool that needs the managed Python runtime (metric compute, query-history SQL analysis) ran on a host that cannot reach `github.com` to download the pinned `uv` and Python | The server still starts and serves catalog and search tools. Restore network access and retry, or pre-build the runtime where network is available: `ktx admin runtime install --yes` |

View file

@ -29,18 +29,9 @@ below.
| `--agents` | Install agent configuration and rules only | `false` |
| `--target <target>` | Agent target: `claude-code`, `claude-desktop`, `codex`, `cursor`, `opencode`, or `universal` | - |
| `--global` | Install agent integration into the global target scope for `claude-code` or `codex` | `false` |
| `--install-dir <path>` | Install project-scoped agent configuration | ktx project dir |
| `--yes` | Accept project creation and runtime install defaults where setup asks for confirmation | `false` |
| `--no-input` | Disable interactive terminal input | - |
> **`--install-dir <path>`**
>
> Installs project-scoped agent configuration into the specified directory.
> The path is resolved against the current directory and created if it doesn't
> exist. Use it to install `.claude/`, `.mcp.json`, and rules where you open
> your agent (for example, `--install-dir .`). This option is mutually exclusive
> with `--global` and `--local`.
Use the global `--project-dir <path>` option when setup should target a
specific directory.
@ -60,9 +51,9 @@ prompts.
| Flag | Description |
|------|-------------|
| `--llm-backend <backend>` | LLM backend: `anthropic`, `vertex`, `claude-code`, or `codex` |
| `--llm-backend <backend>` | LLM backend: `anthropic`, `vertex`, or `claude-code` |
| `--llm-backend claude-code` | Use the local Claude Code session for **ktx** LLM calls |
| `--llm-backend codex` | Use local Codex authentication for **ktx** LLM calls |
| `--llm-model <model>` | LLM model ID or backend model alias to validate and save |
| `--anthropic-api-key-env <name>` | Environment variable containing the Anthropic API key |
| `--anthropic-api-key-file <path>` | File containing the Anthropic API key |
| `--vertex-project <project>` | Vertex AI project ID, `env:NAME`, or `file:/path` reference |
@ -71,17 +62,9 @@ prompts.
Choose only one Anthropic credential source. Anthropic credential flags are only
valid with the Anthropic backend; Vertex flags are only valid with the Vertex
backend. The `claude-code` and `codex` backends use local authentication instead
of Anthropic API key or Vertex flags. After you choose a backend, `ktx setup`
writes that backend's per-role model preset to `ktx.yaml`. To change a model,
edit the matching `llm.models.<role>` value in `ktx.yaml`.
With `--no-input`, `ktx setup` does not assume a default LLM provider, because
every backend needs credentials only you can supply. Pass `--llm-backend`
explicitly. Note that `--target` selects the agent integration, not the LLM
provider: `ktx setup --target claude-code --no-input` still needs
`--llm-backend claude-code` to use your Claude subscription for **ktx** LLM
calls.
backend. The `claude-code` backend uses local Claude Code authentication instead
of Anthropic API key or Vertex flags. For Claude Code, `--llm-model` accepts
`sonnet`, `opus`, `haiku`, or a full Claude model ID.
### Embeddings
@ -120,9 +103,9 @@ runtime features are missing.
| Flag | Description |
|------|-------------|
| `--database <driver>` | Database driver to configure; repeatable. Choices: `sqlite`, `duckdb`, `postgres`, `mysql`, `clickhouse`, `sqlserver`, `bigquery`, `snowflake` |
| `--database <driver>` | Database driver to configure; repeatable. Choices: `sqlite`, `postgres`, `mysql`, `clickhouse`, `sqlserver`, `bigquery`, `snowflake` |
| `--database-connection-id <id>` | Existing selected connection id; repeatable. With `--database` or `--database-url`, connection id for the new connection. |
| `--database-url <url>` | URL, `env:NAME`, or `file:/path` for one new URL-style database connection; also used as the SQLite or DuckDB path |
| `--database-url <url>` | URL, `env:NAME`, or `file:/path` for one new URL-style database connection; also used as the SQLite path |
| `--database-schema <schema>` | Database schema or dataset to include; repeatable |
| `--skip-databases` | Leave database setup incomplete |
@ -134,21 +117,6 @@ incomplete.
MySQL, and SQL Server; `schema_names` for Snowflake; `dataset_ids` for
BigQuery; and `databases` for ClickHouse.
A BigQuery `--database-schema` value may be qualified as `project.dataset` to
scan a dataset hosted in another project (such as
`bigquery-public-data.austin_311`); a bare value stays in the credentials'
project. Setup does not discover foreign-project datasets, so supply qualified
entries explicitly. See
[Primary sources → BigQuery](/docs/integrations/primary-sources#cross-project-datasets).
With `--no-input`, scope for a scope-bearing driver (PostgreSQL, MySQL,
ClickHouse, SQL Server, BigQuery, Snowflake) must come from `--database-schema`
or from existing connection config in `ktx.yaml` (for example
`connections.<id>.dataset_ids`). When neither is set, the database step fails
fast and prints the missing scope flag and config key — non-interactive setup
never auto-discovers and scans every schema. SQLite has no scope and is
unaffected.
### Query History
| Flag | Description |
@ -163,8 +131,8 @@ unaffected.
Query history setup is supported for Postgres, BigQuery, and Snowflake. The
window flag applies to BigQuery and Snowflake; Postgres reads the current
`pg_stat_statements` aggregate data instead of a time-windowed history table.
Later `ktx ingest` runs build enriched context and need a configured model and
embeddings, including when query history is enabled.
Enabling query history makes deep ingest readiness matter for later
`ktx ingest` runs.
When query history is enabled for PostgreSQL, Snowflake, or BigQuery,
`ktx setup` runs a non-blocking readiness probe after the connection test
@ -174,13 +142,6 @@ fix the prerequisite. If the later schema-context build also fails, interactive
setup offers **Disable query history and retry** so you can finish database
setup with `connections.<id>.context.queryHistory.enabled: false`.
After the schema scan completes, setup can derive query-history service-account
filters from in-scope history. If **ktx** finds clear operational roles, it
prints each proposed exclusion with a reason and writes
`connections.<id>.context.queryHistory.filters.serviceAccounts` only when you
apply the proposal. In non-interactive setup with `--yes`, the proposal is
applied automatically. Existing `serviceAccounts` blocks are never overwritten.
For BigQuery, the remediation tells you to grant `roles/bigquery.resourceViewer`
on the BigQuery project, or grant a custom role that contains
`bigquery.jobs.listAll`.
@ -193,15 +154,15 @@ sources. This is equivalent to passing `--skip-sources` in scripted setup.
| Flag | Description |
|------|-------------|
| `--source <type>` | Context-source connector type: `dbt`, `metricflow`, `metabase`, `looker`, `lookml`, `notion`, or `sigma` |
| `--source <type>` | Context-source connector type: `dbt`, `metricflow`, `metabase`, `looker`, `lookml`, or `notion` |
| `--source-connection-id <id>` | Connection id for context-source setup |
| `--source-path <path>` | Local source path for dbt, MetricFlow, or LookML |
| `--source-git-url <url>` | Git URL for dbt, MetricFlow, or LookML |
| `--source-branch <branch>` | Git branch for context-source setup |
| `--source-subpath <path>` | Repo subpath for context-source setup |
| `--source-auth-token-ref <ref>` | `env:` or `file:` credential reference for source repo auth or Notion integration token |
| `--source-auth-token-ref <ref>` | `env:` or `file:` credential reference for source repo auth |
| `--source-url <url>` | Source service URL for Metabase or Looker |
| `--source-api-key-ref <ref>` | `env:` or `file:` API key reference for Metabase |
| `--source-api-key-ref <ref>` | `env:` or `file:` API key reference for Metabase or Notion |
| `--source-client-id <id>` | Looker client id |
| `--source-client-secret-ref <ref>` | `env:` or `file:` Looker client secret reference |
| `--source-warehouse-connection-id <id>` | Warehouse connection id used for context-source mapping |
@ -224,22 +185,12 @@ ktx setup
# Run setup for a specific project directory
ktx setup --project-dir ./analytics
# Use Claude Code for ktx LLM calls
# Use Claude Code with Opus for ktx LLM calls
ktx setup \
--project-dir ./analytics \
--llm-backend claude-code
--llm-backend claude-code \
--llm-model opus
# Configure **ktx** to use local Codex authentication for LLM work
ktx setup --llm-backend codex --no-input
```
When you choose `--llm-backend codex`, setup prints a warning if the public
Codex SDK and CLI surface cannot prove full Claude-Code-style isolation. The
backend restricts **ktx** runtime MCP tools to each run, but Codex may still
load user Codex config and built-in command execution or read-only file
capabilities.
```bash
# Script a Postgres connection that reads its URL from the environment
ktx setup \
--project-dir ./analytics \
@ -270,21 +221,6 @@ ktx setup \
--source-warehouse-connection-id warehouse \
--metabase-database-id 1
# Add a Notion source that crawls selected root pages
ktx setup \
--source notion \
--source-connection-id notion-main \
--source-auth-token-ref env:NOTION_TOKEN \
--notion-crawl-mode selected_roots \
--notion-root-page-id abc123def456
# Add a Sigma source
ktx setup \
--source sigma \
--source-connection-id sigma-main \
--source-client-id your-client-id \
--source-client-secret-ref env:SIGMA_CLIENT_SECRET
# Install project-scoped agent integration for Codex
ktx setup --agents --target codex
```
@ -314,7 +250,6 @@ Use `ktx status` for repeatable readiness checks after setup exits.
|-------|-------|----------|
| Setup resumes an unexpected project | `KTX_PROJECT_DIR` or nearest `ktx.yaml` points to another directory | Pass `--project-dir <path>` explicitly |
| Setup cannot run in CI | Required values are missing and `--no-input` disables prompts | Provide the relevant automation flags or create a fixture `ktx.yaml` |
| `Missing LLM backend: pass --llm-backend …` | `--no-input` setup ran without an LLM backend; `--target` does not select one | Pass `--llm-backend claude-code`, `codex`, `anthropic`, or `vertex` (with that backend's credential flags) |
| Provider health check fails | Provider key, model id, Vertex project, or Vertex location is invalid | Fix the `env:` or `file:` reference and rerun setup |
| Python runtime is missing | The selected setup needs runtime-backed agent, query-history, Looker, or local embedding features | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx admin runtime install` command |
| `--enable-query-history` is rejected | The selected database driver does not support query history | Use Postgres, BigQuery, or Snowflake, or rerun without query-history flags |

View file

@ -11,16 +11,13 @@ the vocabulary agents use to generate correct SQL.
```bash
ktx sl [options] [query...] # list (bare) or search (with query)
ktx sl read <sourceName>
ktx sl validate <sourceName>
ktx sl validate <sourceName> [options]
ktx sl query [options]
```
- Bare `ktx sl` lists semantic sources.
- `ktx sl <query...>` searches semantic sources. Multi-word queries are joined
with a space.
- `ktx sl read <sourceName>` prints the YAML for one source. Add
`--connection-id` only when the source name exists in multiple connections.
- `ktx sl <query...>` searches semantic sources (multi-word queries are
joined with a space).
- `ktx sl validate` and `ktx sl query` remain as explicit subcommands.
## Subcommands
@ -29,7 +26,6 @@ ktx sl query [options]
|-----------|-------------|
| (none, no query) | List semantic sources |
| (none, with query) | Search semantic sources |
| `read <sourceName>` | Print the YAML for one semantic source |
| `validate <sourceName>` | Validate a semantic source against the database schema |
| `query` | Compile or execute a semantic query |
@ -44,23 +40,17 @@ ktx sl query [options]
| `--output <mode>` | Output mode: `pretty` (default in TTY), `plain` (TSV), or `json` | `pretty` |
| `--json` | Shortcut for `--output=json` (overrides `--output`) | `false` |
### `sl read`
| Flag | Description | Default |
|------|-------------|---------|
| `--connection-id <id>` | Optional **ktx** connection id for disambiguation | - |
### `sl validate`
| Flag | Description | Default |
|------|-------------|---------|
| `--connection-id <id>` | Optional **ktx** connection id for disambiguation | - |
| `--connection-id <id>` | **ktx** connection id (required) | - |
### `sl query`
| Flag | Description | Default |
|------|-------------|---------|
| `--connection-id <id>` | Required **ktx** connection id | - |
| `--connection-id <id>` | **ktx** connection id | - |
| `--query-file <path>` | JSON semantic query file | - |
| `--measure <measure>` | Measure to query; repeatable (at least one required) | - |
| `--dimension <dimension>` | Dimension to include; repeatable | - |
@ -75,9 +65,8 @@ ktx sl query [options]
| `--no-input` | Disable interactive managed runtime installation | - |
| `--max-rows <n>` | Maximum rows to return when executing | - |
`sl query` requires `--connection-id` and at least one `--measure` unless
`--query-file` is set. `--query-file` must point to a JSON semantic query
object.
`sl query` requires at least one `--measure` unless `--query-file` is set.
`--query-file` should point to a JSON semantic query object.
## Examples
@ -94,16 +83,7 @@ ktx sl --json
# Search sources as JSON
ktx sl "revenue" --json
# Print the YAML for a source name that is unique across connections
ktx sl read orders
# Print the YAML for a source name that exists in multiple connections
ktx sl --connection-id my-warehouse read orders
# Validate a source name that is unique across connections
ktx sl validate orders
# Validate a source name that exists in multiple connections
# Validate a source against the live schema
ktx sl validate orders --connection-id my-warehouse
# Compile a query and view the generated SQL
@ -164,12 +144,6 @@ shows `#1`, `#2`, and later rank badges for the displayed results. Plain and
JSON output keep the raw `score` value, which is a ranking score rather than a
percentage.
`ktx sl read <sourceName>` prints the source YAML directly to stdout when the
source name is unique across connections. If the name exists in multiple
connections, rerun the command with `--connection-id <id>`. The command does
not wrap output in pretty, plain, or JSON formatting, so it can be piped to
other tools.
```json
{
"sql": "SELECT orders.status, SUM(orders.total_amount) AS total_revenue FROM public.orders GROUP BY orders.status",
@ -186,8 +160,7 @@ other tools.
| Error | Cause | Recovery |
|-------|-------|----------|
| Source not found | Source name or connection id is wrong | Run `ktx sl <query>` or `ktx sl --connection-id <id>` to find the exact source name, then retry `ktx sl read <sourceName>` or `ktx sl validate <sourceName>` |
| Source name is ambiguous | The same source name exists in multiple connections | Rerun with `--connection-id <id>` from the error message |
| Source not found | Source name or connection id is wrong | Run `ktx sl --json` and retry with an exact source name and connection id |
| Validation fails | YAML references missing columns, invalid joins, or invalid SQL expressions | Fix the source YAML and rerun `ktx sl validate` |
| Query compile fails | Measure, dimension, filter, or segment name is invalid | Search sources with `ktx sl <query>`, inspect the source YAML in your project files, then retry using declared fields |
| Execution returns too many rows | `--max-rows` is missing or too high | Add `--max-rows` with a bounded value before executing |

View file

@ -21,7 +21,7 @@ ktx status [options]
| `--json` | Print JSON output | `false` |
| `-v`, `--verbose` | Show every check, including passing ones | `false` |
| `--validate` | Only validate the `ktx.yaml` schema; skip readiness checks | `false` |
| `--fast` | Skip checks that require external communication (query-history readiness probes, Claude Code auth probe, and Codex auth probe) | `false` |
| `--fast` | Skip checks that require external communication (query-history readiness probes and Claude Code auth probe) | `false` |
| `--no-input` | Disable interactive terminal input | - |
## Examples
@ -39,7 +39,7 @@ ktx status --verbose
# Validate ktx.yaml without running readiness checks
ktx status --validate
# Skip slow probes (query-history readiness, Claude Code auth, Codex auth)
# Skip slow probes (query-history readiness, Claude Code auth)
ktx status --fast
# Check a project from another directory
@ -57,16 +57,6 @@ flow, then rerun `ktx status`. Use `--fast` to skip this probe (useful in CI
or offline contexts); skipped checks render as `-` and carry
`"status": "skipped"` in JSON output.
For `llm.provider.backend: codex`, `ktx status` runs a minimal non-interactive
Codex request. If the probe fails, authenticate Codex locally with the Codex CLI
and verify the Codex CLI installation.
When `llm.provider.backend: codex` is configured, `ktx status` also prints a
warning when the installed public Codex SDK and CLI surface cannot prove full
Claude-Code-style isolation. The warning does not block authenticated Codex
usage, but it marks the project status as partial so you can make an explicit
runtime-isolation decision.
A `Local data` section summarises what the project has accumulated locally:
ingest run counts, last completed timestamp per connection, knowledge page
counts by scope, semantic-layer source and dictionary value counts, and the
@ -94,6 +84,6 @@ stats, and are always shown (they do not require external communication).
|-------|-------|----------|
| No **ktx** project found | Current directory has no `ktx.yaml` and `KTX_PROJECT_DIR` is unset | `ktx status` runs setup checks; run from a **ktx** project or set `KTX_PROJECT_DIR` for project checks |
| Project config check fails | The project directory is missing or has an invalid `ktx.yaml` | Run `ktx setup` to resume setup |
| Schema validation fails | A field **ktx** recognizes has an invalid value. Unrecognized keys are reported as non-blocking warnings (exit `0`), not failures | Run `ktx status --validate --json` for structured issue details, then edit `ktx.yaml` or rerun `ktx setup` |
| Schema validation fails | `ktx.yaml` does not match the current config schema | Run `ktx status --validate --json` for structured issue details, then edit `ktx.yaml` or rerun `ktx setup` |
| Semantic search check warns | Embeddings are not configured or the provider probe failed | Run `ktx setup` or inspect the check's `fix` field in JSON output |
| Query history check warns | A database has query history enabled but the warehouse prerequisites are missing | Fix the warehouse extension, grants, or history access, then rerun `ktx status` |

View file

@ -1,24 +1,21 @@
---
title: "ktx wiki"
description: "List, search, or read wiki pages."
description: "List or search wiki pages."
---
List, search, and read wiki pages in your **ktx** project. Wiki pages are
Markdown documents that capture business definitions, rules, and gotchas.
Agents search them for context when answering questions about your data.
List and search wiki pages in your **ktx** project. Wiki pages are Markdown
documents that capture business definitions, rules, and gotchas. Agents search
them for context when answering questions about your data.
## Command signature
```bash
ktx wiki [options] [query...] # list (bare) or search (with query)
ktx wiki read <key>
ktx wiki [options] [query...]
```
- Bare `ktx wiki` lists local wiki pages.
- `ktx wiki <query...>` searches local wiki pages. Multi-word queries are
joined with a space.
- `ktx wiki read <key>` prints the whole Markdown file for one wiki page,
including YAML frontmatter.
- `ktx wiki <query...>` searches local wiki pages (multi-word queries are
joined with a space).
Edit the Markdown files under `wiki/` directly, or ingest source content with
`ktx ingest`, when you need to add or update wiki knowledge.
@ -28,17 +25,10 @@ Edit the Markdown files under `wiki/` directly, or ingest source content with
| Flag | Description | Default |
|------|-------------|---------|
| `--user-id <id>` | Local user id | `local` |
| `-c, --connection <id>` | Scope results to one connection: unscoped pages plus pages tagged with that connection | - |
| `--limit <number>` | Maximum search results (search mode only) | - |
| `--output <mode>` | Output mode: `pretty` (default in TTY), `plain` (TSV), or `json` | `pretty` |
| `--json` | Shortcut for `--output=json` (overrides `--output`) | `false` |
`-c, --connection <id>` takes a connection id from the `connections` map in
`ktx.yaml` (an unknown id is rejected). It narrows both list and search to
pages that are not tied to any connection plus pages tagged with that
connection, so an agent working against one database sees only the wiki
knowledge relevant to it.
`ktx wiki <query>` uses hybrid search when `storage.search` is `sqlite-fts5`.
**ktx** combines lexical SQLite FTS5 matches, token matches, and semantic matches
from wiki page embeddings stored in `.ktx/db.sqlite`. If embeddings are not
@ -57,18 +47,9 @@ ktx wiki --json
# Search wiki pages
ktx wiki "monthly recurring revenue"
# List pages scoped to one connection (unscoped + connection-tagged)
ktx wiki --connection warehouse
# Search within one connection's scope
ktx wiki "monthly recurring revenue" -c warehouse
# Search wiki pages as JSON
ktx wiki "monthly recurring revenue" --json --limit 10
# Print the exact Markdown file for a known page key
ktx wiki read revenue-definitions
# Print search results as TSV
ktx wiki "monthly recurring revenue" --output plain
@ -81,10 +62,8 @@ ktx --debug wiki "monthly recurring revenue" --json
Wiki commands print clack-style pretty output in a TTY and TSV-style plain
output when requested. JSON output wraps the items with a command metadata
envelope. Search results include `matchReasons` and `lanes` metadata so you can
see whether lexical, token, or semantic search contributed to the ranking. Use
`ktx wiki read <key>` when you need the full page contents. Read output is the
exact Markdown file stored on disk, including YAML frontmatter, and is not
wrapped in pretty, plain, or JSON formatting.
see whether lexical, token, or semantic search contributed to the ranking. Open
the matching Markdown files directly when you need the full page contents.
Pretty search output shows `#1`, `#2`, and later rank badges for the displayed
results. Plain and JSON output keep the raw `score` value, which is a ranking
score rather than a percentage.
@ -142,4 +121,4 @@ stays machine-readable:
| Error | Cause | Recovery |
|-------|-------|----------|
| Search returns no results | The query terms do not match summaries, tags, or content, and the semantic lane is unavailable or has no positive matches | Run with `--debug`, check the semantic lane status, retry with business synonyms, then create a page if the knowledge is missing |
| A page is missing | No Markdown file exists for that business context or `ktx wiki read <key>` used the wrong key | Run `ktx wiki <query>` to find the page key, then retry `ktx wiki read <key>` |
| A page is missing | No Markdown file exists for that business context | Add a file under `wiki/` or run `ktx ingest <connectionId>` |

View file

@ -36,11 +36,9 @@ ktx
wiki
list
search <query>
read <key>
sl
list
search <query>
read <sourceName>
validate <sourceName>
query
sql
@ -59,7 +57,6 @@ ktx
stop
status
reindex
completion <shell>
```
The public context-build entrypoint is `ktx ingest [connectionId]` or
@ -74,56 +71,6 @@ The public context-build entrypoint is `ktx ingest [connectionId]` or
| `-v`, `--version` | Show the CLI package name and version. |
| `-h`, `--help` | Show help for the current command. |
## Update notices
> **Note:** The update notifier writes only to stderr and keeps command stdout
> unchanged.
When a newer package is available on your installed release channel, `ktx`
prints a short notice after the command finishes:
```text
↑ Update available: ktx 0.9.0 → 0.10.0
npm i -g @kaelio/ktx
```
Stable installs compare against the npm `latest` dist-tag.
Release-candidate installs compare against the `next` dist-tag and show:
```text
npm i -g @kaelio/ktx@next
```
The check is skipped for JSON output, CI, non-TTY stdout, and hidden completion
commands. To opt out explicitly, set any of these environment variables:
```bash
KTX_NO_UPDATE_CHECK=1
NO_UPDATE_NOTIFIER=1
DO_NOT_TRACK=1
```
The `ktx` CLI prints one npm command because globally installed binaries don't
expose a reliable runtime package-manager signal. If you prefer another global
package manager, use the equivalent command:
```bash
pnpm add -g @kaelio/ktx
yarn global add @kaelio/ktx
```
## Build-view star prompt
During an interactive context build, `ktx setup` and `ktx ingest` can show a dim
GitHub star reminder above the `Ctrl+C to stop` hint. **ktx** skips this prompt
for CI, non-TTY output, and `DO_NOT_TRACK=1`.
To suppress only this prompt while keeping other notices enabled, set:
```bash
KTX_NO_STAR=1
```
## Project resolution
Most commands are project-aware. Pass `--project-dir <path>` when scripting or
@ -150,10 +97,6 @@ ktx ingest
ktx sl "revenue"
ktx wiki "revenue recognition"
# Print a known wiki page or semantic source
ktx wiki read revenue-definitions
ktx sl --connection-id warehouse read orders
# Execute read-only SQL
ktx sql --connection warehouse "select count(*) from public.orders"

View file

@ -11,7 +11,6 @@
"ktx-wiki",
"ktx-status",
"ktx-mcp",
"ktx-admin",
"ktx-completion"
"ktx-admin"
]
}

View file

@ -1,111 +0,0 @@
---
title: AI Resources
description: How coding agents read, cite, and act on the ktx docs - Markdown endpoints, a task router, and copy-paste prompts.
---
This page is for coding assistants that read or cite the **ktx** docs. It covers
the machine-readable endpoints, a task router for common questions, and
copy-paste prompts. It is scoped to documentation lookup and safe command
discovery - to wire **ktx** into an agent client, see
[Agent Clients](/docs/integrations/agent-clients).
## Markdown endpoints
**ktx** docs are available as plain Markdown so assistants never have to parse
the rendered HTML site.
- [`/llms.txt`](/llms.txt) - a curated index of high-value pages and agent entry
points. **Start here.**
- [`/llms-full.txt`](/llms-full.txt) - the entire docs corpus in one response.
Use only when a task needs broad context across many pages.
- **Per-page Markdown** - append `.md` to any docs URL:
```text
https://docs.kaelio.com/ktx/docs/getting-started/quickstart.md
https://docs.kaelio.com/ktx/docs/cli-reference/ktx-sl.md
https://docs.kaelio.com/ktx/docs/guides/building-context.md
```
A request for any docs URL with an `Accept: text/markdown` header returns the
same Markdown without the `.md` suffix:
```bash
curl -H "Accept: text/markdown" https://docs.kaelio.com/ktx/docs/getting-started/quickstart
```
Each Markdown response leads with the page title, description, canonical URL, and
Markdown URL; frontmatter is stripped; code blocks and tables are preserved; and
missing pages return a plain-text `404` instead of falling back to HTML. Rendered
pages also expose a **Copy as Markdown** action near the title.
### Retrieval order
1. Fetch [`/llms.txt`](/llms.txt).
2. Pick one or two relevant per-page `.md` URLs.
3. Fetch [`/llms-full.txt`](/llms-full.txt) only when page-level docs are not
enough.
```bash
curl https://docs.kaelio.com/ktx/llms.txt
curl https://docs.kaelio.com/ktx/docs/getting-started/quickstart.md
```
## Task router
| User asks the agent to explain... | Read first | Then read |
|------------------------------------|------------|-----------|
| What **ktx** does | [Introduction](/docs/getting-started/introduction) | [The Context Layer](/docs/concepts/the-context-layer) |
| How to start from a checkout | [Quickstart](/docs/getting-started/quickstart) | [ktx setup](/docs/cli-reference/ktx-setup) |
| How to check project readiness | [ktx status](/docs/cli-reference/ktx-status) | [Quickstart](/docs/getting-started/quickstart) |
| How context gets built | [Building Context](/docs/guides/building-context) | [ktx ingest](/docs/cli-reference/ktx-ingest) |
| How semantic YAML works | [Writing Context](/docs/guides/writing-context) | [ktx sl](/docs/cli-reference/ktx-sl) |
| How machine-readable CLI output is shaped | [ktx sl](/docs/cli-reference/ktx-sl) | [ktx wiki](/docs/cli-reference/ktx-wiki) |
## Agent instructions
Paste this into a project or system prompt when an assistant needs to answer
from the **ktx** docs:
```text
When answering ktx docs questions:
1. Start with https://docs.kaelio.com/ktx/llms.txt.
2. Fetch the smallest relevant Markdown page (append .md to its docs URL).
3. Prefer the .md route over rendered HTML.
4. Use https://docs.kaelio.com/ktx/llms-full.txt only when the task needs broad docs context.
5. Quote commands exactly from docs pages.
6. If docs and local CLI behavior disagree, say what differs and prefer local verified output.
```
## Prompts
Replace project names, connection ids, and business terms with your own values.
**Install and configure ktx in a project**
```text
Run npx skills add Kaelio/ktx --skill ktx and use the ktx skill to install and configure ktx
```
**Find the right command**
```text
Find the correct ktx command for this task: <task>. Start with /llms.txt, then fetch the smallest relevant CLI reference .md page. Quote the exact command and flags from the docs.
```
**Review semantic changes**
```text
Review the ktx semantic-layer and wiki changes in this branch. Check that measures have clear definitions, joins use valid keys, hidden or internal columns are not exposed to agents, and validation passes. List concrete file and line issues first.
```
## Guardrails
- Do not invent CLI flags - fetch the relevant CLI reference page.
- Do not scrape rendered HTML when a `.md` route exists.
- Do not treat `/llms.txt` as complete documentation - use it as an index, then
fetch the linked pages.
- Do not include credentials or secrets in prompts, URLs, or copied docs
snippets.
- When docs and local CLI behavior disagree, prefer the local CLI output and
mention the mismatch.

View file

@ -1,5 +1,5 @@
{
"title": "Community & Resources",
"title": "Community",
"defaultOpen": true,
"pages": ["support", "contributing", "telemetry", "ai-resources"]
"pages": ["support", "contributing", "telemetry"]
}

View file

@ -1,15 +1,12 @@
---
title: Telemetry
description: Understand what usage telemetry ktx collects and how to opt out.
description: Understand what anonymous usage telemetry ktx collects and how to opt out.
---
**ktx** collects aggregated usage telemetry so maintainers can see
which commands work, where setup fails, and which parts of the data-agent
workflow need improvement. Telemetry is opt-out: it turns on the first time you
run **ktx** in any way — an interactive command, a script, or an
agent-launched MCP server — and prints a one-time notice (to the terminal when
there is one, otherwise to standard error). It stays disabled in CI and whenever
an opt-out is set.
**ktx** collects anonymous, aggregated usage telemetry from interactive CLI
runs so maintainers can see which commands work, where setup fails, and which
parts of the data-agent workflow need improvement. Telemetry is opt-out and
disabled automatically in CI and non-interactive runs.
## Opt out
@ -20,58 +17,23 @@ Use any of these mechanisms to disable telemetry:
| `export KTX_TELEMETRY_DISABLED=1` | Disables telemetry for the shell and child processes |
| `export DO_NOT_TRACK=1` | Standard do-not-track environment variable |
| `CI=1` | Automatic in CI |
| Edit `~/.ktx/telemetry.json` and set `"enabled": false` | Persistent for the machine, including the MCP server |
| Non-TTY output | Automatic for pipes and scripts |
| Edit `~/.ktx/telemetry.json` and set `"enabled": false` | Persistent for the machine |
## What we collect
High-level signals: which commands run, how long they take, whether they
High-level signals only: which commands run, how long they take, whether they
succeed or fail, and basic environment metadata (CLI version, Node version, OS
platform). When an operation fails, we also include diagnostic detail about the
error so we can debug it. For project-level analysis, **ktx** sends a salted
hash of the project directory to group events.
When an agent reaches **ktx** through MCP, we also record the connecting client
tool's self-reported name and version (for example Claude Desktop, Cursor, or
Cline) so we can see which agents people use **ktx** with. That describes the
tool, never you or your data.
platform). For project-level analysis, **ktx** sends a salted hash of the
project directory — never the raw path.
## What we never collect
We build telemetry around counts and coarse signals, not the contents of your
data or configuration. We don't deliberately collect your `ktx.yaml`, query
results, passwords, API keys, or access tokens.
The one place environment-specific text can appear is failure diagnostics: when
an operation errors, the detail we record is the error as your tools reported
it, which can include identifiers from your setup. If you'd rather send nothing
at all, turn telemetry off using any of the options above.
## Error reports
When telemetry is enabled, **ktx** sends PostHog Error Tracking `$exception`
events for CLI and daemon exceptions. Error reports help group crashes and
handled failures into PostHog issues.
Error reports can include:
- Stack frames, including function names, local file paths, line numbers, and
SDK-provided source context.
- Error class names and raw error messages.
- Cause chains when the runtime exposes them.
- `source`, `handled`, and `fatal` diagnostic fields.
- Runtime version, OS, architecture, and CI fields.
- The hashed `projectId` when **ktx** knows the project.
Error reports never intentionally include:
- Secrets, credentials, API keys, tokens, cookies, signed URLs, or auth headers.
- Database URLs, connection strings, DSNs, raw argv, or raw environment values.
- SQL text, schema names, table names, or column names as explicit payload
properties.
- Customer row data.
- User prompt text or raw MCP arguments.
The same opt-out controls listed above disable error reports.
- File paths, hostnames, environment variable values, or command arguments
- `ktx.yaml` contents, connection passwords, API keys, or tokens
- Schema names, table names, column names, SQL text, or query results
- Error messages or stack traces
- Git remote URLs, Git user email, OS user, or hostname
## Storage and retention

View file

@ -1,161 +0,0 @@
---
title: Cross-database federation
description: How ktx federates postgres, mysql, sqlite, and duckdb connections so a single read-only SQL query can join across them without copying data.
---
Cross-database federation lets a single read-only SQL query join tables that
live in different databases. **ktx** achieves this by embedding DuckDB and
using its `ATTACH` mechanism to connect each member database read-only. The
join executes inside DuckDB at query time — live data, no ETL, no copy.
You run federated queries as raw SQL against the `_ktx_federated` connection
(see [Querying the federated connection
directly](#querying-the-federated-connection-directly)). Semantic-layer queries
(`ktx sl query` / the `sl_query` tool) stay per-connection; pointing one at
`_ktx_federated` returns an error telling you to use raw SQL instead.
Federation activates automatically when a `ktx.yaml` file declares two or more
attach-compatible connections. There is nothing to configure and no federation
block to add. With zero or one compatible connection the behavior is unchanged.
## Which connections participate
The v1 federation engine supports four drivers:
| Driver | Participates in federation |
|--------|---------------------------|
| `postgres` | Yes |
| `mysql` | Yes |
| `sqlite` | Yes |
| `duckdb` | Yes |
| `snowflake` | No — standalone connection |
| `bigquery` | No — standalone connection |
| `clickhouse` | No — standalone connection |
| `sqlserver` | No — standalone connection |
Non-participating connections continue to work exactly as they did. They are
queried independently; they do not appear as federation members.
## How it activates
**ktx** inspects the connections in `ktx.yaml` at startup. When it finds two or
more connections whose driver is `postgres`, `mysql`, `sqlite`, or `duckdb`, it
instantiates the DuckDB federation engine and attaches each one read-only.
There is no `federation:` key, no opt-in flag, and no connection-level setting
to enable. The engine is derived entirely from what is already declared.
A minimal `ktx.yaml` that triggers federation:
```yaml
connections:
pg_books:
driver: postgres
url: "postgres://user:pass@localhost:5432/books" # pragma: allowlist secret
sqlite_reviews:
driver: sqlite
path: ./data/reviews.db
```
Two attach-compatible connections are present, so federation is active.
## Table naming in federated queries
Inside a federated query, postgres and mysql tables use a three-part name:
`connectionId.schema.table`. SQLite and DuckDB tables use the two-part form
`connectionId.table`, since ktx addresses both as single-namespace members. In
both cases the connection's `id` field in `ktx.yaml` becomes the catalog name
inside DuckDB.
If a connection `id` is not a bare SQL identifier — for example it contains a
hyphen, like `books-db` — double-quote it in the query the same way DuckDB
quotes any identifier: `"books-db".public.books`. Writing it unquoted
(`books-db.public.books`) is a SQL syntax error, not a federation feature.
For the example above:
- `pg_books.public.books` — the `books` table in the `public` schema of the
postgres connection
- `sqlite_reviews.reviews` — the `reviews` table in the sqlite connection
These fully qualified names are what you write when you query the federated
connection with raw SQL (see [Querying the federated connection
directly](#querying-the-federated-connection-directly)). A source file's own
`table:` field is not prefixed this way — see [Source files keep member-native
table refs](#source-files-keep-member-native-table-refs) below.
## Source names in the federated view
When you list or search semantic-layer sources under the federated connection,
each source's `name` is prefixed with its member connection id — for example
`pg_books.books` and `sqlite_reviews.reviews`. The prefix keeps names unique
when two members own a table with the same name: a `users` table in each of
`pg_app` and `sqlite_app` surfaces as `pg_app.users` and `sqlite_app.users`
rather than colliding on a bare `users`.
## Source files keep member-native table refs
A source file's physical `table:` field is not prefixed with the connection id.
It stays the member-native reference the connector uses on its own —
`public.books` for the postgres member, `reviews` for the sqlite member —
because the same file backs a per-connection semantic-layer query against that
member, which runs on the member's own driver where a `pg_books.` catalog prefix
would point at a database that does not exist. The connection-id prefix is a
DuckDB catalog name that appears only in raw federated SQL; the member prefix on
the source `name` (above) is independent of it.
## Cross-database joins
Write a cross-database join as raw SQL against `_ktx_federated` — see
[Querying the federated connection
directly](#querying-the-federated-connection-directly) below for a runnable
example. DuckDB attaches both members and resolves the join live at query time.
Declaring the join in a source file's `joins:` block is not supported yet. The
semantic layer plans each connection on its own, so a `joins:` entry whose `to:`
points at a table in another member is not resolved across the federation
boundary. Until that lands, express cross-database joins as raw SQL.
## Querying the federated connection directly
The federated connection is addressable by its id,
`_ktx_federated`, anywhere **ktx** runs read-only SQL. The same id works for the
`ktx sql` command and for a data agent calling the `sql_execution` MCP tool, so
both surfaces can run a cross-database query without a source file:
```bash
ktx sql -c _ktx_federated \
"SELECT b.title, avg(r.rating) AS avg_rating
FROM pg_books.public.books b
JOIN sqlite_reviews.reviews r ON b.id = r.book_id
GROUP BY b.title"
```
Table names follow the rules from
[Table naming in federated queries](#table-naming-in-federated-queries):
three-part `connectionId.schema.table` for postgres and mysql, two-part
`connectionId.table` for sqlite and duckdb. The `_ktx_federated` id is virtual —
it is never written to `ktx.yaml` and only exists when two or more attach-compatible
connections are declared. It surfaces in `ktx connection` and in the agent's
connection list so the id is discoverable. Querying a single member database
directly with its own connection id (`ktx sql -c pg_books ...`) is unchanged.
If any member connection sets
[`query_policy: semantic-layer-only`](/docs/configuration/ktx-yaml#query-policy),
raw SQL against `_ktx_federated` is rejected as a whole: a federated query can
touch any member's tables, so one restricted member restricts the federation.
## Federated queries are read-only
DuckDB attaches every member database with read-only access. Federated queries
are `SELECT`/`WITH` only. No writes, no DDL, and no mutations reach any member
database through the federation engine.
## Current limitations
- **Raw SQL joins only.** Cross-database joins are written as raw SQL; declaring
them in a source's `joins:` block and automatic discovery of cross-database
relationships are not available yet. Intra-database relationship discovery for
each member connection is unchanged.
- **postgres, mysql, sqlite, and duckdb only.** Other drivers (snowflake,
bigquery, clickhouse, sqlserver) do not participate in federation in this
version. They remain usable as standalone connections.

View file

@ -1,5 +1,5 @@
{
"title": "Concepts",
"defaultOpen": true,
"pages": ["the-context-layer", "semantic-layer-internals", "cross-database-federation", "wiki-retrieval"]
"pages": ["the-context-layer", "semantic-layer-internals", "wiki-retrieval"]
}

View file

@ -46,7 +46,7 @@ read, how to think, and where to put the results.
</p>
<ul className="mt-3 space-y-2 text-sm leading-6 text-fd-foreground">
<li><code className="text-[13px] font-semibold">llm</code> - provider, models, prompt cache</li>
<li><code className="text-[13px] font-semibold">ingest</code> - connectors, embeddings, work units</li>
<li><code className="text-[13px] font-semibold">ingest</code> - adapters, embeddings, work units</li>
<li><code className="text-[13px] font-semibold">scan</code> - enrichment, relationships</li>
<li><code className="text-[13px] font-semibold">agent</code> - research-agent feature flags</li>
</ul>
@ -66,9 +66,8 @@ read, how to think, and where to put the results.
## Minimal config
A working `ktx.yaml` needs one entry in `connections`. Everything else accepts
defaults. The example below registers a local Postgres connection; building
context with `ktx ingest warehouse` also needs a model and embeddings, which
`ktx setup` configures.
defaults. The example below is enough for `ktx ingest warehouse` to run a fast
schema scan against a local Postgres.
```yaml
connections:
@ -109,7 +108,6 @@ context-source drivers share the map.
| `postgres` | Warehouse | `driver` | `url`, `enabled_tables`, `historicSql`, `context.queryHistory` |
| `mysql` | Warehouse | `driver` | `url`, `enabled_tables` |
| `sqlite` | Warehouse | `driver` | `url` or `path`, `enabled_tables` |
| `duckdb` | Warehouse | `driver` | `url` or `path`, `enabled_tables` |
| `sqlserver` | Warehouse | `driver` | `url`, `enabled_tables` |
| `bigquery` | Warehouse | `driver` | `credentials_json`, `dataset_ids`, `enabled_tables`, `historicSql` |
| `snowflake` | Warehouse | `driver` | `schema_names`, `enabled_tables`, `historicSql` |
@ -120,16 +118,13 @@ context-source drivers share the map.
| `dbt` | Context source | `driver`, one of `source_dir` or `repo_url` | `branch`, `path`, `profiles_path`, `target`, `project_name` |
| `metricflow` | Context source | `driver`, `metricflow.repoUrl` | `metricflow.branch`, `metricflow.path`, `metricflow.auth_token_ref` |
| `notion` | Context source | `driver`, `auth_token_ref` | `crawl_mode`, `root_*_ids`, `max_*_per_run` |
| `sigma` | Context source | `driver`, `client_id`, `client_secret_ref` | `api_url` |
### Warehouse drivers
Warehouse connections are open objects: the listed fields are validated, and
any other field is preserved and passed through to the connector. Use
`enabled_tables` to scope ingest to a specific list of objects - useful for
smoke tests. Each entry accepts a `catalog.db.name`, `db.name`, or bare `name`
qualifier. ktx restricts the scan to the listed objects and fails with a clear
error (naming the available objects) if none match.
`enabled_tables` to scope deep ingest to a specific list of
`schema.table` names - useful for smoke tests.
```yaml
connections:
@ -141,18 +136,6 @@ connections:
- public.customers
```
For SQLite, which exposes a single `main` schema, the qualified `main.<name>`
and the bare `<name>` forms select the same object:
```yaml
connections:
local-db:
driver: sqlite
path: ./warehouse.db
enabled_tables:
- customers # equivalent to main.customers
```
Connector-specific scope fields let setup and scan use the same warehouse
boundary:
@ -174,12 +157,6 @@ connections:
dataset_ids: [analytics, mart]
```
A BigQuery `dataset_ids` / `dataset_id` entry may be written `project.dataset`
to introspect a dataset hosted in another project (for example
`bigquery-public-data.austin_311`); jobs still bill to the `project_id` in
`credentials_json`. A bare `dataset` keeps using your own project. See
[Primary sources → BigQuery](/docs/integrations/primary-sources#cross-project-datasets).
For Postgres, MySQL, SQL Server, and Snowflake connections, set
`maxConnections` when scan or ingest work needs to stay below the target's
connection cap. Postgres, MySQL, and SQL Server default to `10`; Snowflake
@ -201,57 +178,9 @@ connections:
context:
queryHistory:
enabled: true
enabledSchemas:
- orbit_raw
- orbit_analytics
minExecutions: 5
```
- `enabledSchemas`: Optional list of schema or dataset names that query-history
ingest may mine. Omit it to let **ktx** derive the modeled schema floor from
the connection and semantic-layer sources. Use `["*"]` to disable the floor
for discovery runs.
- `filters.serviceAccounts`: Optional service-account filter block. During
setup, when query history is enabled and no service-account block already
exists, **ktx** can propose exact role patterns such as `^svc_loader$` from
observed in-scope query history. The block uses `mode: exclude` and remains
hand-editable.
### Query policy
Set `query_policy: semantic-layer-only` on a warehouse connection to stop
agents from authoring SQL against it. The default, `read-only-sql`, allows
parser-validated read-only SQL through `ktx sql` and the `sql_execution` MCP
tool alongside semantic-layer queries.
```yaml
connections:
warehouse:
driver: snowflake
query_policy: semantic-layer-only
```
With `semantic-layer-only`:
- `ktx sql` and the `sql_execution` MCP tool reject the connection with a
clear error. When every SQL connection in the project is restricted, the
`sql_execution` tool is not registered at all.
- Raw SQL against the federated connection (`_ktx_federated`) is rejected
when any member connection is restricted.
- Semantic-layer queries (`ktx sl query`, the `sl_query` tool) accept only
measures predefined in the semantic-layer sources. Composed aggregate
expressions such as `sum(orders.amount)` are rejected wherever they appear,
including inside `filters` (a `HAVING`-style clause may only compare a
predefined measure by name, e.g. `orders.revenue > 100`). Grouping by
declared dimensions, filtering on columns, and segments remain available.
- `connection_list` marks the connection as restricted so agents route to
`sl_query` instead of burning a failed call.
The policy governs agent-facing query authorship, not data access: **ktx**'s
own scan, ingest, and semantic-layer-generated SQL still run, and context
tools such as `entity_details` and `dictionary_search` still expose schema
metadata and sampled values.
### Metabase
```yaml
@ -382,31 +311,6 @@ connections:
| `max_knowledge_creates_per_run` | Max new wiki pages created per run (0-25). |
| `max_knowledge_updates_per_run` | Max existing wiki pages updated per run (0-100). |
### Sigma
```yaml
connections:
sigma-main:
driver: sigma
api_url: https://api.sigmacomputing.com
client_id: "<your-client-id>"
client_secret_ref: env:SIGMA_CLIENT_SECRET
workbookFilter:
includeArchived: false
includeExplorations: false
updatedSince: "2026-01-01T00:00:00Z"
```
| Field | Purpose |
|-------|---------|
| `api_url` | Sigma API base URL. Defaults to `https://api.sigmacomputing.com` (GCP US). Override for AWS US (`https://aws-api.sigmacomputing.com`) or other regions. |
| `client_id` | Sigma OAuth client ID. Required. |
| `client_secret` / `client_secret_ref` | Literal secret or reference. Prefer the `_ref`. |
| `connectionMappings` | Maps Sigma internal connection UUIDs to **ktx** warehouse connection IDs. Enables `sl_validate` for projected semantic-layer sources. |
| `workbookFilter.includeArchived` | Include archived workbooks during ingest. Default: `false`. |
| `workbookFilter.includeExplorations` | Include exploration workbooks during ingest. Default: `false`. |
| `workbookFilter.updatedSince` | ISO 8601 date string. Only workbooks updated on or after this date are fetched. Useful for limiting ingest scope at large scale. |
## `setup`
Captured by the setup wizard. The only field **ktx** still reads is
@ -426,14 +330,15 @@ setup:
## `storage`
`storage` controls where **ktx** keeps its own state and search index. Defaults
work for a single-user local project.
`storage` controls where **ktx** keeps its own state and search index, and how
state changes are committed. Defaults work for a single-user local project.
```yaml
storage:
state: sqlite # sqlite | postgres
search: sqlite-fts5 # sqlite-fts5 | postgres-hybrid
git:
auto_commit: true
author: "ktx <ktx@example.com>"
```
@ -441,7 +346,8 @@ storage:
|-------|------|---------|---------|
| `state` | `sqlite` \| `postgres` | `sqlite` | Backend for ktx state. `sqlite` uses `.ktx/db.sqlite`; `postgres` expects a configured Postgres connection. |
| `search` | `sqlite-fts5` \| `postgres-hybrid` | `sqlite-fts5` | Backend for search indexes. `postgres-hybrid` combines lexical and vector search in Postgres. |
| `git.author` | `string` | `ktx <ktx@example.com>` | Git author identity for commits. Standard `Name <email>` form. |
| `git.auto_commit` | `boolean` | `true` | When `true`, ktx auto-commits changes to the git-backed state store. |
| `git.author` | `string` | `ktx <ktx@example.com>` | Git author identity for auto-commits. Standard `Name <email>` form. |
## `llm`
@ -457,10 +363,6 @@ llm:
models:
default: claude-sonnet-4-6
triage: claude-haiku-4-5
candidateExtraction: claude-sonnet-4-6
curator: claude-opus-4-7
reconcile: claude-opus-4-7
repair: claude-haiku-4-5
promptCaching:
enabled: true
systemTtl: 1h
@ -473,28 +375,13 @@ llm:
| Field | Type | Default | Purpose |
|-------|------|---------|---------|
| `provider.backend` | `none` \| `anthropic` \| `vertex` \| `gateway` \| `claude-code` \| `codex` | `none` | Selected backend. `none` disables LLM features. `claude-code` uses the local Claude Code session and needs no API key. `codex` uses local Codex authentication and needs no API key. |
| `provider.backend` | `none` \| `anthropic` \| `vertex` \| `gateway` \| `claude-code` | `none` | Selected backend. `none` disables LLM features. `claude-code` uses the local Claude Code session and needs no API key. |
| `provider.anthropic.api_key` | `string` | - | Anthropic API key. Required when `backend: anthropic`. Accepts `env:` or `file:` references. |
| `provider.anthropic.base_url` | `string` | - | Override the Anthropic API base URL (proxy, self-hosted gateway). |
| `provider.gateway.api_key` / `base_url` | `string` | - | Credentials for an AI Gateway provider. Required when `backend: gateway`. |
| `provider.vertex.project` | `string` | - | Google Cloud project ID hosting the Vertex AI endpoint. |
| `provider.vertex.location` | `string` | - | Vertex AI region (for example `us-east5`). Required when the `vertex` block is present. |
Use `codex` when local Codex authentication should power **ktx** LLM work:
```yaml
llm:
provider:
backend: codex
models:
default: gpt-5.5
triage: gpt-5.5
candidateExtraction: gpt-5.5
curator: gpt-5.5
reconcile: gpt-5.5
repair: gpt-5.5
```
### Model roles
`models` overrides the per-role model. Keys are fixed; values are
@ -522,7 +409,7 @@ provider-specific model identifiers.
## `ingest`
`ingest` controls how **ktx** builds context from your stack. It lists the
connectors to run, the embedding provider used when connectors embed documents,
adapters to run, the embedding provider used when adapters embed documents,
and the concurrency and failure policy for work units.
```yaml
@ -541,24 +428,14 @@ ingest:
stepBudget: 40
maxConcurrency: 2
failureMode: continue
rateLimit:
enabled: true
throttleThreshold: 0.8
minConcurrencyUnderPressure: 1
maxWaitMs: 600000
retry:
maxAttempts: 6
baseDelayMs: 1000
maxDelayMs: 60000
jitter: true
```
### Connectors
### Adapters
`adapters` is a list of connector IDs that should run. Each ID matches a
`adapters` is a list of adapter IDs that should run. Each ID matches a
connector that **ktx** ships locally:
| Connector ID | What it ingests |
| Adapter ID | What it ingests |
|------------|-----------------|
| `live-database` | Live warehouse introspection (schemas, tables, columns, samples). |
| `historic-sql` | Query history from Postgres `pg_stat_statements`, BigQuery `INFORMATION_SCHEMA.JOBS`, or Snowflake query history. |
@ -568,7 +445,7 @@ connector that **ktx** ships locally:
| `looker` | Looker dashboards and looks via the API. |
| `metabase` | Metabase cards, dashboards, and database mappings. |
| `notion` | Notion pages and databases for wiki context. |
| `fake` | Test/demo connector. Useful in fixtures. |
| `fake` | Test/demo adapter. Useful in fixtures. |
### Embeddings
@ -597,24 +474,6 @@ handles failures.
| `workUnits.maxConcurrency` | `int > 0` | `1` | How many work units run in parallel. |
| `workUnits.failureMode` | `abort` \| `continue` | `continue` | `abort` stops the whole ingest run on the first failure; `continue` records it and keeps going. |
### Rate limits
`rateLimit` controls provider-neutral pacing for LLM calls during ingest. When a
provider reports a subscription window, retry-after delay, or HTTP 429,
**ktx** pauses new work-unit model calls, shows a transient wait in the CLI,
and reduces work-unit concurrency while the provider is under pressure.
| Field | Type | Default | Purpose |
|-------|------|---------|---------|
| `rateLimit.enabled` | `boolean` | `true` | Master switch for ingest LLM rate-limit pacing and visible waits. |
| `rateLimit.throttleThreshold` | `number between 0 and 1` | `0.8` | Fraction of a known provider window at which **ktx** starts reducing concurrency. |
| `rateLimit.minConcurrencyUnderPressure` | `int > 0` | `1` | Effective work-unit concurrency while a provider is under rate-limit pressure. |
| `rateLimit.maxWaitMs` | `int > 0` | unset | Caps how long a single provider-reset wait can last. This bounds each wait, not the whole run: after a capped wait elapses **ktx** retries and may pause again. Omit to wait until the provider's reset time. |
| `rateLimit.retry.maxAttempts` | `int > 0` | `6` | Maximum attempts for a single rate-limited LLM call before the failure surfaces (counts the first try). Also bounds how far opaque backoff grows for responses without a reset time or retry-after value. |
| `rateLimit.retry.baseDelayMs` | `int > 0` | `1000` | Initial opaque retry delay in milliseconds. |
| `rateLimit.retry.maxDelayMs` | `int > 0` | `60000` | Maximum opaque retry delay in milliseconds. |
| `rateLimit.retry.jitter` | `boolean` | `true` | Add jitter to opaque retry delays. |
## `scan`
`scan` configures how schema-level inputs become structured context:
@ -636,7 +495,6 @@ scan:
profileConcurrency: 4
validationConcurrency: 4
validationBudget: all
detectionBudgetMs: 600000
```
### Enrichment
@ -665,7 +523,6 @@ the manifest.
| `relationships.profileConcurrency` | `int > 0` | `4` | Parallel relationship-profile queries against the database. For pooled connectors, effective database concurrency is also bounded by the connection's `maxConnections`. |
| `relationships.validationConcurrency` | `int > 0` | `4` | Parallel relationship validation queries against the database. |
| `relationships.validationBudget` | `all` \| `int ≥ 0` | runtime default | Cap on validation queries per scan. `all` means unlimited. |
| `relationships.detectionBudgetMs` | `int > 0` | `600000` | Wall-clock budget (ms) for the whole relationship-detection stage, checked at table-profile, candidate-validation, and composite-probe boundaries. On exhaustion the stage stops scheduling new work and writes the joins found so far, marked partial; descriptions and embeddings are already durable. Sits above the per-query deadline. Raise it to trigger a fresher, fuller run. |
## `agent`
@ -690,6 +547,19 @@ agent:
| `run_research.max_iterations` | `int ≥ 0` | `20` | Maximum tool-call iterations per research run. |
| `run_research.default_toolset` | `string[]` | `[sl_query, wiki_search, sl_read_source]` | Tool identifiers exposed to the research agent. |
## `memory`
`memory` controls the agent memory subsystem.
```yaml
memory:
auto_commit: true
```
| Field | Type | Default | Purpose |
|-------|------|---------|---------|
| `auto_commit` | `boolean` | `true` | When `true`, ktx auto-commits memory updates to the git-backed store. |
## A full example
Combining the blocks above:
@ -714,17 +584,13 @@ storage:
state: sqlite
search: sqlite-fts5
git:
auto_commit: true
author: "ktx <ktx@example.com>"
llm:
provider:
backend: claude-code
models:
default: sonnet
triage: haiku
candidateExtraction: sonnet
curator: opus
reconcile: opus
repair: haiku
ingest:
adapters:
- live-database
@ -746,25 +612,17 @@ scan:
agent:
run_research:
enabled: true
memory:
auto_commit: true
```
## Validating your config
**ktx** validates `ktx.yaml` when it loads, and treats two kinds of problems
differently:
- **An invalid value on a field ktx recognizes** (for example
`llm.provider.backend: nope`) is a hard error. Setup and CLI commands stop and
report the exact path so you can fix it.
- **An unrecognized key** — one left over from a different **ktx** version, or a
typo such as `scan.relationships.acceptThreshhold` — is tolerated, not fatal.
**ktx** ignores the key and keeps running, so a misspelled field quietly falls
back to its default instead of taking effect. `ktx status` lists each ignored
key as a warning (and exits `0`) so you can remove or correct it when
convenient.
Warehouse connections accept extra driver-specific fields, so passthrough values
like `historicSql` and `context.queryHistory` are allowed.
**ktx** validates `ktx.yaml` strictly: unknown keys at the top level or inside
strict blocks cause setup and CLI commands to fail with a precise path
(`scan.relationships.acceptThreshhold: Unrecognized key`). Warehouse
connections accept extra driver-specific fields, so passthrough values like
`historicSql` and `context.queryHistory` are allowed.
To re-validate without running anything else:

View file

@ -4,7 +4,6 @@ description: ktx is an open-source, self-improving context layer for data agents
---
import { ProductMechanics } from "@/components/product-mechanics";
import { ProductRuntime } from "@/components/product-runtime";
<div className="not-prose mb-10">
<div>
@ -24,7 +23,7 @@ import { ProductRuntime } from "@/components/product-runtime";
>
Make analytics context usable by agents
</h1>
<p className="mt-4 max-w-full text-lg text-fd-muted-foreground" style={{ lineHeight: '1.7' }}>
<p className="mt-4 max-w-2xl text-lg text-fd-muted-foreground" style={{ lineHeight: '1.7' }}>
{'ktx is an open-source context layer for data agents. It turns warehouse metadata, BI tool definitions, query history, docs, and approved metric definitions into reviewable files agents can search and execute.'}
</p>
</div>
@ -60,8 +59,6 @@ serves that context to agents at runtime.
<ProductMechanics />
<ProductRuntime />
## Use it for
Use **ktx** when agents need more than raw database access. Agents can search wiki
@ -95,8 +92,8 @@ best first step for users; contributor setup lives in the community docs.
<Card title="CLI Reference" href="/docs/cli-reference/ktx">
Complete flag and subcommand reference for every **ktx** command.
</Card>
<Card title="AI Resources" href="/docs/community/ai-resources">
Machine-readable docs, a task router, and copy-paste agent prompts.
<Card title="Agent Quickstart" href="/docs/ai-resources/agent-quickstart">
Machine-readable docs and agent-facing setup notes.
</Card>
</Cards>

View file

@ -9,8 +9,8 @@ This guide takes a local analytics project from empty to agent-ready. You'll
install the CLI, run one guided setup command, and hand the context to a
coding assistant.
If you're a coding assistant choosing a docs route, start with
[AI Resources](/docs/community/ai-resources) instead.
If you're a coding assistant choosing a docs route, start with the
[Agent Quickstart](/docs/ai-resources/agent-quickstart) instead.
<div
className="not-prose my-8 overflow-hidden rounded-2xl border"
@ -30,18 +30,17 @@ If you're a coding assistant choosing a docs route, start with
</div>
<div className="mt-2.5 text-base leading-relaxed text-fd-foreground">
Try **ktx** against a real data stack - Postgres, dbt, Metabase, and Notion
pre-loaded with the Orbit demo corpus. Hit **copy agent setup** on the page
for a one-shot prompt that has an agent install the full four-source demo,
or grab the raw credentials to paste into `ktx setup` yourself.
pre-loaded with the Orbit demo corpus. The page lists demo credentials
you can paste straight into `ktx setup`.
</div>
<a
href="https://www.kaelio.com/start"
href="https://kaelio.com/start"
className="group mt-5 inline-flex items-center gap-2 rounded-full px-4 py-2.5 text-sm font-semibold text-white no-underline shadow-[inset_0_1px_0_rgba(255,255,255,0.35),0_2px_4px_rgba(255,138,77,0.2),0_10px_24px_-8px_rgba(255,138,77,0.55)] transition-all duration-200 hover:-translate-y-0.5 hover:shadow-[inset_0_1px_0_rgba(255,255,255,0.4),0_3px_6px_rgba(255,138,77,0.28),0_16px_30px_-8px_rgba(255,138,77,0.65)]"
style={{
background: 'linear-gradient(180deg, #ff9d63 0%, #f97316 100%)',
}}
>
Get demo credentials at www.kaelio.com/start
Get demo credentials at kaelio.com/start
<svg
width="14"
height="14"
@ -191,12 +190,6 @@ Install the published package globally:
npm install -g @kaelio/ktx
```
To upgrade an existing install later, re-run with the `@latest` tag:
```bash
npm install -g @kaelio/ktx@latest
```
**ktx** is open source. If you'd like to hack on it or run from a local checkout,
the source lives at [github.com/kaelio/ktx](https://github.com/kaelio/ktx) -
see [Contributing](/docs/community/contributing) to get set up.
@ -218,12 +211,11 @@ The wizard walks you through everything **ktx** needs in one pass:
3. **Embeddings** - picks an embeddings backend. Choose OpenAI for hosted
embeddings or `sentence-transformers` to run locally without an API key.
4. **Database** - adds at least one primary connection. Supported drivers:
PostgreSQL, Snowflake, BigQuery, MySQL, ClickHouse, SQL Server, SQLite, and
DuckDB.
SQLite, PostgreSQL, MySQL, SQL Server, BigQuery, and Snowflake.
5. **Context sources** - optionally adds dbt, MetricFlow, LookML, Looker,
Metabase, or Notion. You can skip and add them later.
6. **Build** - offers to run the first ingest so semantic sources and wiki
pages are ready for agents. If you skip it, build later with `ktx ingest`.
6. **Build** - runs the first ingest so semantic sources and wiki pages
are ready for agents.
7. **Agent integration** - installs project-local rules for Claude Code,
Codex, Cursor, OpenCode, or universal `.agents`.
@ -243,7 +235,7 @@ Testing warehouse
Connection test passed
Building schema context for warehouse
Running database scan
Running fast database ingest
```
If setup exits early, rerun `ktx setup` in the same directory. **ktx** keeps
@ -254,18 +246,6 @@ progress under `.ktx/setup/` and resumes from the remaining work.
> resuming setup, connecting an agent, checking status, or exploring a
> pre-built demo project.
When the wizard finishes, it states where you stand and the single next action:
- **Context built** - **ktx** confirms it is ready for agents and points you to
open your coding agent and ask a data question.
- **Build skipped** - **ktx** tells you setup is complete and that the only step
left is to build context with `ktx ingest`.
Re-running `ktx setup` on an already-configured project goes straight to the
remaining step - building context or connecting an agent - instead of
re-asking every question. Once everything is ready, it confirms you are set
rather than reopening the configuration menu.
## Verify
When setup finishes, check readiness:
@ -287,41 +267,18 @@ Agent integration ready: yes (codex:project)
For a structured check inside scripts, use `ktx status --json`.
If you skipped the build, `ktx context built` shows `no`. Build it with
`ktx ingest` - there is no need to re-run `ktx setup`.
When setup finishes building context, its final context check looks like:
When setup builds deep context, its final context check looks like:
```text
ktx context is ready for agents.
Databases:
warehouse: database context complete
warehouse: deep context complete
Context sources:
dbt_main: memory update complete
```
Before the build starts, **ktx** runs a live test for every connection the
build depends on. A context build can take several minutes, so if any required
connection is unreachable or misconfigured the build is blocked up front and
**ktx** names the failing connection by id and connector type:
```text
ktx cannot build context: a required connection failed its live test.
Failed connections:
warehouse (postgres)
Each connection must be reachable before ktx builds context.
Run `ktx connection test <id>` to see the error, fix the connection, then retry.
```
Run `ktx connection test <connection-id>` to see the underlying error, fix the
connection, then continue. In interactive setup you can retry without
restarting; with `--no-input` the build exits non-zero and names the failing
connection so scripts can stop early.
## Connect a coding agent
The setup wizard installs project-local agent rules in the last step. To
@ -339,16 +296,6 @@ separate `ktx` binary on `PATH`. If the CLI path changes, rerun
## What setup writes
**ktx** writes plain files so people and agents can review changes in git.
**ktx** initializes a git repository at the project directory and writes context
changes there. If the project directory is nested inside another repository,
**ktx** still keeps its own repo and does not commit to the parent repo.
Because **ktx** owns that repository, it will not adopt one it did not create. If
you point setup at a directory that is already a git repository's root - such as
an existing application checkout - **ktx** stops and asks you to pick a dedicated
directory instead. In the setup wizard choose the **New subfolder** option (for
example `ktx-project`), or pass a fresh `--project-dir` when running setup
non-interactively.
| Path | Purpose |
|------|---------|
@ -378,7 +325,7 @@ ktx setup \
Then build context:
```bash
ktx ingest warehouse
ktx ingest warehouse --fast
```
See [ktx setup](/docs/cli-reference/ktx-setup) for the full automation flag
@ -391,8 +338,7 @@ surface.
| `ktx: command not found` | Reinstall `@kaelio/ktx` and open a new shell |
| Setup resumes the wrong project | Pass `--project-dir <path>` |
| LLM or embeddings health check fails | Rerun setup and pick a different credential, model, or backend |
| Database test fails | Use the setup recovery menu to retry or re-enter details; if it still fails, verify the same connection with the database's native client |
| Context build blocked: a connection failed its live test | Run `ktx connection test <connection-id>` to see the error, fix the connection, then retry the build |
| Database test fails | Verify the same connection with the database's native client, then rerun setup |
| Agent integration is incomplete | Run `ktx setup --agents --target <target>` |
## Next steps

View file

@ -24,9 +24,7 @@ external metadata can attach to known warehouse tables.
## Database ingest
Database ingest always builds enriched context: tables, columns, types,
constraints, and row counts, plus AI-generated descriptions, embeddings, and
relationship evidence.
Database ingest records table, column, type, constraint, and row-count context.
```bash
# Build one configured database connection
@ -36,37 +34,37 @@ ktx ingest warehouse
ktx ingest --all
```
Enriched ingest needs a configured model and embeddings. Run `ktx setup` first;
connections without that configuration fail before any work starts.
Depth controls how much context **ktx** builds:
Local-auth backends keep provider credentials out of `ktx.yaml`:
| Flag | Best for | What it does |
|------|----------|--------------|
| `--fast` | First setup, quick refreshes, CI smoke checks | Deterministic fast ingest with tables, columns, types, constraints, and row counts |
| `--deep` | Agent-ready context for real analysis | Fast ingest plus deep enrichment with descriptions, embeddings, relationship evidence, and optional query history |
Examples:
```bash
ktx setup --llm-backend claude-code --no-input
ktx setup --llm-backend codex --no-input
ktx ingest warehouse --fast
ktx ingest warehouse --deep
ktx ingest --all --deep
```
With `claude-code`, **ktx** agent loops can invoke only the **ktx** MCP tools
for the current run. With `codex`, **ktx** restricts the temporary runtime MCP
server to the current run's tool set, disables Codex web search, requests a
read-only sandbox, and sets `approval_policy=never`. The public Codex SDK and
CLI surface may still load user Codex config and built-in command execution or
read-only file capabilities, so use `claude-code` for stricter runtime tool
isolation.
Deep ingest needs LLM and embedding readiness. Otherwise run `ktx setup` or use
`--fast`.
With `claude-code`, **ktx** agent loops can invoke only the **ktx** MCP tools for the
current run.
## Query history
PostgreSQL, BigQuery, and Snowflake can add query-history context: common joins,
filters, redaction rules, high-usage templates, and service-account exclusions.
When query history is enabled during setup, **ktx** reviews observed in-scope
roles and can write exact `filters.serviceAccounts` patterns for operational
traffic such as loader or refresh roles.
filters, service-account patterns, redaction rules, and high-usage templates.
Enable it during setup, store it under `connections.<id>.context.queryHistory`,
or request it for one run:
```bash
ktx ingest warehouse --query-history
ktx ingest warehouse --deep --query-history
# Set the lookback window for BigQuery or Snowflake query history
ktx ingest warehouse --query-history-window-days 30
```
@ -76,8 +74,8 @@ for one run.
## Relationship evidence
**ktx** scores relationship candidates during database ingest. The public CLI
does not expose separate relationship review subcommands.
**ktx** scores relationship candidates during supported deep database ingest. The
public CLI does not expose separate relationship review subcommands.
## Context-source ingest
@ -102,7 +100,6 @@ Supported source types:
| `looker` | Looker API | Explores, looks, dashboards, and model metadata |
| `metabase` | Metabase API | Questions, dashboards, table metadata, and mappings |
| `notion` | Notion API | Wiki pages and business knowledge |
| `sigma` | Sigma API | Data model specs, pages, element metadata, and workbook metadata |
Context-source ingest writes semantic source YAML and wiki Markdown, reconciling
with local edits.
@ -162,7 +159,7 @@ After interactive setup:
```bash
ktx status
ktx ingest --all
ktx ingest --all --deep
ktx status
```
@ -179,8 +176,8 @@ ktx wiki "revenue" --json --limit 10
| Symptom | Likely cause | Recovery |
|---------|--------------|----------|
| Connection not configured | The connection id is missing from `ktx.yaml` | Add it with `ktx setup` |
| Enrichment is not configured | LLM or embeddings are not setup-ready | Run `ktx setup` to configure a model and embeddings |
| Query history is unsupported | The selected database driver does not expose query history | Run ingest without query-history flags |
| Deep readiness is missing | LLM or embeddings are not setup-ready | Run `ktx setup`, or rerun with `--fast` |
| Query history is unsupported | The selected database driver does not expose query history | Run fast ingest without query-history flags |
| No connections configured | The project has no entries under `connections` | Run `ktx setup` and add a database or context-source connection |
| Context-source flags have no effect | Query-history flags were supplied for a context-source connector | Use query-history flags only for database connections |
| Context-source flags have no effect | Depth and query-history flags were supplied for a context-source connector | Use those flags only for database connections |
| Text ingest stops early | `--fail-fast` stopped on the first failed item | Fix the item or rerun without `--fail-fast` |

View file

@ -16,7 +16,6 @@ Set `llm.provider.backend` to one of these values:
- `gateway`: Use AI Gateway-compatible Anthropic model ids.
- `claude-code`: Use your local Claude Code session through the Claude Agent
SDK. **ktx** strips provider-routing environment variables from child processes.
- `codex`: Use your local Codex authentication through the Codex SDK.
## Claude Code
@ -30,65 +29,24 @@ llm:
default: sonnet
triage: haiku
candidateExtraction: sonnet
curator: opus
reconcile: opus
repair: haiku
curator: sonnet
reconcile: sonnet
repair: sonnet
```
During setup, choose the backend interactively or pass it in automation:
During setup, choose the backend interactively or pass the model in automation:
```bash
ktx setup --llm-backend claude-code --no-input
ktx setup --llm-backend claude-code --llm-model opus --no-input
```
Setup writes `sonnet`, `haiku`, and `opus` aliases into `llm.models`. You can
edit any role to another alias or a full Claude model ID after setup.
For Claude Code, `sonnet`, `opus`, and `haiku` map to **ktx** defaults. Full Claude
model IDs are also accepted.
`claude-code` exposes only **ktx** MCP tools for the current agent loop. SDK init
metadata may still list host slash commands, skills, and subagents; **ktx** does not
grant execution access to them.
## Codex backend
Use `codex` when you want **ktx** to run LLM-backed workflows through your
local Codex authentication instead of a direct provider API key.
```yaml
llm:
provider:
backend: codex
models:
default: gpt-5.5
triage: gpt-5.5
candidateExtraction: gpt-5.5
curator: gpt-5.5
reconcile: gpt-5.5
repair: gpt-5.5
```
Configure it non-interactively:
```bash
ktx setup --llm-backend codex --no-input
```
This is separate from Codex agent-client setup. `ktx setup --agents --target
codex` installs instructions and MCP access for an end-user Codex session.
`ktx setup --llm-backend codex` makes **ktx** itself execute ingest, scan
enrichment, memory, and other LLM-backed work through Codex.
During runtime loops, **ktx** starts a temporary loopback MCP server for the
current run, exposes only the tools passed to that run, asks Codex to use a
read-only sandbox, sets `approval_policy=never`, auto-approves only those
run-scoped MCP tools, and disables Codex web search.
Codex backend isolation is currently limited by the public Codex SDK and CLI
surface. Codex may still load user Codex config and built-in command execution
or read-only file capabilities. Use `llm.provider.backend: claude-code` when
you need stricter Claude-Code-style runtime tool isolation, or remove host
Codex MCP and tool config before running untrusted prompts through the `codex`
backend.
## Prompt caching
`llm.promptCaching` has partial parity on `claude-code`. Status and doctor warn

View file

@ -61,14 +61,11 @@ committing the file.
## A typical review session
The loop above describes the shape. Run these commands from the **ktx** project
directory. **ktx** keeps that directory as its own git repository, even when the
directory lives inside another repository, so reviewing context changes never
requires committing to a parent application repo.
The loop above describes the shape. In practice, one review session looks like
this:
```bash
# 1. Run ingest on a branch
cd /path/to/ktx-project
git checkout -b ingest/2026-05-21
ktx ingest --all

View file

@ -111,13 +111,12 @@ non-obvious terms.
Agents can refresh context when the user asks them to:
```bash
ktx ingest warehouse
ktx ingest warehouse --fast
ktx ingest
ktx ingest --file docs/revenue-notes.md --connection-id warehouse
```
Database ingest builds enriched context and requires a configured model and
embeddings; run `ktx setup` first if they are not ready.
Use `--deep` only when LLM and embedding setup is ready.
## Good agent behavior

View file

@ -44,17 +44,12 @@ Use this order for most context changes:
Semantic sources are YAML files for queryable tables or custom SQL. They define
agent-facing measures, dimensions, segments, joins, and grain.
Semantic source files live under:
Semantic source files live at:
```text
semantic-layer/<connection-id>/
semantic-layer/<connection-id>/<source-name>.yaml
```
The file's `name:` field is the source's identity — it carries the warehouse
identifier verbatim, including case. The filename is a derived label: simple
lowercase names get `<source-name>.yaml`, anything else gets a slugged
filename. Renaming a file does not rename the source.
### Minimal source
```yaml
@ -157,7 +152,7 @@ joins:
| Field | Required | Description |
|-------|----------|-------------|
| `name` | Yes | Source identity (not the filename). When overlaying an ingested table, match the manifest identifier verbatim, including case (e.g. `SIGNED_UP`); for a new standalone source, lowercase words and underscores are recommended. |
| `name` | Yes | Source identifier. Use lowercase words and underscores. |
| `descriptions` | No | Description map keyed by source, such as `user`, `dbt`, or `ai`. |
| `table` or `sql` | Yes | Database table or custom SQL expression. Use exactly one. |
| `grain` | Yes | Columns that uniquely identify a row at the source grain. |
@ -321,23 +316,6 @@ Useful frontmatter:
5. Add `sl_refs` for relevant semantic sources.
6. Search again with a user-like phrase.
### Ingest an authoritative document verbatim
When the document is already the source of truth — a metric-definition sheet, a
formula spec, a runbook, compliance text — you want **ktx** to index and surface
it, not re-author it. Instead of hand-copying the file into `wiki/global/`, ingest
it verbatim:
```bash
ktx ingest --file docs/rfm-bucket-definitions.md --verbatim
```
The body is stored byte-for-byte (the LLM only derives `summary`, `tags`, and
`sl_refs` for the absent frontmatter fields), the page key is derived from the
filename, and re-running is a safe no-op. Existing frontmatter — including fields
**ktx** does not model, like `effective_date` — passes through unchanged. See
[`ktx ingest`](/docs/cli-reference/ktx-ingest) for the full flag reference.
## Review context changes
Before accepting agent-written context:

View file

@ -68,30 +68,19 @@ If you choose an install mode, it then asks which targets to install:
```
When at least one selected target supports project-scoped setup, the command
asks where to install agent config:
When every selected target supports both project and global setup, the command
also asks where to install supported agent config:
```txt
◆ Where should ktx install agent config?
◆ Where should ktx install supported agent config?
│ ktx project: /path/to/your/ktx-project
│ ○ ktx project directory /path/to/your/ktx-project
│ ○ Current directory /path/to/where/you/ran/ktx
│ ○ Custom directory… (enter a path)
│ ○ Project scope (ktx project directory)
│ ○ Global scope (user config)
```
The first three choices write project-scoped files (`.claude/`, `.mcp.json`,
`.cursor/`, skills, and rules) into the chosen directory while still pointing
them at this ktx project. Use **Current directory** or **Custom directory…**
when you open your coding agent from somewhere other than the ktx project
directory. **Current directory** is hidden when it is already the ktx project
directory, and **Global scope** appears only when every selected target
supports global setup. Non-interactive runs pass `--install-dir <path>` (for
example `--install-dir .`) for the same result.
## Generated files
**ktx** writes MCP client configuration and analytics guidance by default. It writes

View file

@ -1,6 +1,6 @@
---
title: Context Sources
description: Ingest semantic context from dbt, MetricFlow, LookML, Metabase, Looker, Notion, Sigma, and Google Drive.
description: Ingest semantic context from dbt, MetricFlow, LookML, Metabase, Looker, and Notion.
---
Context sources feed your existing analytics tooling into **ktx**. During ingestion, **ktx** extracts metadata from each source and uses a reconciliation agent to reconcile it with your existing semantic layer and knowledge base - preserving accepted edits rather than overwriting.
@ -27,7 +27,7 @@ LookML uses top-level `repoUrl`, and MetricFlow uses nested
| Field | Required | Description |
|-------|----------|-------------|
| `driver` | Yes | Source connector: `dbt`, `metricflow`, `lookml`, `metabase`, `looker`, `notion`, `sigma`, or `gdrive` |
| `driver` | Yes | Source connector: `dbt`, `metricflow`, `lookml`, `metabase`, `looker`, or `notion` |
| `source_dir` | For local file sources | Absolute or project-relative source directory |
| `repo_url` | For Git-hosted dbt sources | Git repository URL |
| `repoUrl` | For Git-hosted LookML sources | Git repository URL |
@ -38,16 +38,15 @@ LookML uses top-level `repoUrl`, and MetricFlow uses nested
## dbt
Ingests schema definitions, model descriptions, column metadata, and column test definitions from a dbt project.
Ingests schema definitions, model descriptions, column metadata, and test coverage from a dbt project.
### What it provides
- Model and source definitions from `schema.yml` files
- Column names, descriptions, and data types
- Column tests, mapped to semantic facts — `not_null` / `unique` become column constraints, `accepted_values` becomes enum value lists, and `relationships` becomes join / foreign-key edges
- Model and source tags, and source freshness settings
MetricFlow `semantic_models:` and `metrics:` are ingested through the separate [MetricFlow](#metricflow) source, not the dbt driver.
- Column descriptions and types
- Test coverage signals
- Semantic model references (if using dbt semantic layer)
- Data lineage between models
### Connection config
@ -88,9 +87,9 @@ connections:
### What gets ingested
- **Semantic-layer overlays** (`semantic-layer/*.yaml`): descriptions, constraints, enum values, and joins from the dbt YAML are written onto the semantic source for the matching warehouse table. Overlays land on the warehouse connection that owns the table, which is usually a different connection than the dbt source itself.
- **Wiki pages** (`wiki/`): for definitions or relationships that don't map to a confirmed physical table.
- **Work units** for parallel processing: one per schema file under `models/` when the project has more than 25 YAML files, otherwise a single combined unit.
- YAML semantic sources generated from dbt schema files
- One work unit per semantic source (for projects with >25 YAML files) or all at once for smaller projects
- Column descriptions, tests, and relationships are preserved
---
@ -102,7 +101,7 @@ Ingests MetricFlow semantic models and metric definitions. Useful when your team
- Semantic model definitions (entities, dimensions, measures)
- Cross-model metric definitions
- Entity relationships between models, inferred from matching foreign and primary entities
- Dimension and entity relationships between models
### Connection config
@ -134,7 +133,7 @@ For a local path:
### What gets ingested
- Semantic models with their entities, dimensions, measures, and the join edges inferred from entity relationships
- Semantic models with their entities, dimensions, and measures
- Metric definitions with their expressions and filters
- Work units organized by connected component (metrics + related semantic models grouped together)
@ -179,10 +178,10 @@ For a local path:
### What gets ingested
- One work unit per model, plus a unit for orphan views and one per dashboard
- Semantic-layer sources per view — overlays for thin `sql_table_name` wrappers, standalone sources for `derived_table` views
- Measures, joins (with their Looker `relationship:`), and field types mapped to column types (`yesno` → boolean, date/timestamp → time)
- Wiki pages for relationships and descriptions, with warehouse identifiers verified before writing
- View and model definitions organized by connected component
- LookML field types mapped to semantic layer column types
- Join definitions and relationship cardinalities
- SQL table references for warehouse mapping validation
### Warehouse mapping
@ -193,19 +192,19 @@ Optionally validate that LookML references match your expected Looker connection
expectedLookerConnectionName: postgres_connection
```
This compares each model's `connection:` declaration against the expected name. Mismatched models are flagged, and semantic-layer writes are disabled for them during that ingest while wiki extraction still proceeds.
This validates that LookML model `connection:` declarations match expectations, flagging mismatches during ingestion.
---
## Metabase
Ingests collections, questions, models, and metrics — with their underlying SQL — from a Metabase instance. Maps Metabase databases to your **ktx** warehouse connections.
Ingests dashboards, questions, and their underlying SQL queries from a Metabase instance. Maps Metabase databases to your **ktx** warehouse connections.
### What it provides
- Collections and their hierarchy, used to organize ingested context
- Questions, models, and metrics — resolved SQL for both native and structured (MBQL) queries
- Each card's output schema: column types and primary/foreign-key hints
- Dashboard metadata and organization
- Question/query definitions (native SQL and structured queries)
- Table and column usage patterns from queries
- Database-to-warehouse relationship mapping
### Connection config
@ -234,9 +233,9 @@ Generate an API key in Metabase: **Admin > Settings > Authentication > API Keys*
### What gets ingested
- Semantic-layer sources generated from each card's resolved SQL and column metadata, written to the mapped warehouse connection
- Fallback wiki notes only when a referenced table can't be mapped or an identifier can't be verified
- One work unit per Metabase collection; re-syncs reprocess only collections with changed cards
- Semantic sources generated from SQL queries in questions
- Wiki pages for dashboards (purpose, key metrics, relationships)
- Work units per dashboard and per question
### Warehouse mapping
@ -290,10 +289,10 @@ Generate API credentials in Looker: **Admin > Users > Edit > API Keys**.
### What gets ingested
- Semantic-layer sources from explore fields, written to the mapped warehouse connection (mapped explores only)
- Wiki pages capturing reusable metric, segment, and domain knowledge from dashboards and Looks
- Usage and recency signals that drive a triage gate, focusing processing on high-value content
- Work units per explore, per dashboard, and per Look
- Semantic sources from explore field definitions
- Wiki pages for dashboards (purpose, audience, key metrics)
- Triage signals for automated content classification
- Work units per explore and per dashboard
### Warehouse mapping
@ -315,10 +314,10 @@ Ingests pages and databases from a Notion workspace as wiki pages. Useful for ca
### What it provides
- Notion pages crawled from selected roots or all accessible content
- Page bodies and blocks normalized to Markdown
- Page hierarchy and cross-page links (child pages, mentions, relations)
- Notion databases and their data-source rows as individual pages
- Wiki pages synthesized from Notion content
- Page hierarchy and relationships
- Database schemas (when Notion databases describe primary sources)
- Semantic clustering for organized ingestion
### Connection config
@ -357,7 +356,6 @@ Create an integration at [notion.so/my-integrations](https://www.notion.so/my-in
| `crawl_mode` | `all_accessible` or `selected_roots` | - |
| `root_page_ids` | Page IDs to crawl from (for `selected_roots`) | `[]` |
| `root_database_ids` | Database IDs to include | `[]` |
| `root_data_source_ids` | Data-source IDs to include (for `selected_roots`) | `[]` |
| `max_pages_per_run` | Pages processed per sync | `1000` |
| `max_knowledge_creates_per_run` | New pages created per sync | `25` |
| `max_knowledge_updates_per_run` | Pages updated per sync | `20` |
@ -365,178 +363,17 @@ Create an integration at [notion.so/my-integrations](https://www.notion.so/my-in
### What gets ingested
- Wiki pages synthesized from Notion content (not raw copies)
- Semantic-layer sources when a page defines a reusable dataset or metric mapped to a confirmed non-Notion target; otherwise the fact stays wiki-only
- Page-relevance triage that skips transient content (task lists, status updates, date-titled snapshots)
- Work units clustered by embedding similarity for efficient synthesis
- Domain context extracted and organized by topic
- Triage signals for classifying page relevance
- Work units clustered by semantic similarity for efficient processing
### Notes
- Notion is wiki-first: it writes durable wiki pages by default and only emits semantic-layer sources for content mapped to a confirmed non-Notion target; unmapped facts stay wiki-only
- Notion is knowledge-only - it does not produce semantic layer sources
- Rate limits apply; large workspaces may require multiple ingestion runs
- Incremental sync cursors are stored in `.ktx/db.sqlite`; don't add
`last_successful_cursor` to `ktx.yaml`
---
## Sigma
Ingests data model definitions and workbook metadata from a Sigma workspace as semantic context. Uses the Sigma REST API to fetch data model specs and workbook summaries.
### What it provides
- Data model names, folder paths, and ownership metadata
- Page and element definitions within each data model
- Column identifiers and data types where available
- Workbook names, paths, descriptions, and version metadata
### Connection config
```yaml title="ktx.yaml"
connections:
sigma-main:
driver: sigma
api_url: https://api.sigmacomputing.com # Omit for GCP US (default)
client_id: "<your-client-id>"
client_secret_ref: env:SIGMA_CLIENT_SECRET
```
For the AWS US region, override `api_url`:
```yaml title="ktx.yaml"
connections:
sigma-main:
driver: sigma
api_url: https://aws-api.sigmacomputing.com
client_id: "<your-client-id>"
client_secret_ref: env:SIGMA_CLIENT_SECRET
```
### Authentication
| Method | Config |
|--------|--------|
| OAuth client credentials | `client_id` + `client_secret_ref: env:SIGMA_CLIENT_SECRET` |
Generate a client in Sigma: **Administration → Developer Access → Add New Client**.
### What gets ingested
- Active data model specs, organized by folder into work units
- Workbook metadata (name, path, description, version) — archived and exploration workbooks excluded by default
- Models backed by CSV uploads or unsupported connector subtypes are listed in the manifest but skipped during spec fetch (a Sigma API limitation)
### Warehouse connection mapping
`connectionMappings` is optional. Without it, **ktx** produces wiki knowledge only — no semantic-layer sources are written and warehouse validation is skipped. To get semantic-layer output and enable `sl_validate`, map each Sigma internal connection UUID to a **ktx** warehouse connection ID:
```yaml title="ktx.yaml"
connections:
sigma-main:
driver: sigma
client_id: "<your-client-id>"
client_secret_ref: env:SIGMA_CLIENT_SECRET
connectionMappings:
"<sigma-internal-uuid>": snowflake-prod # data models using this connection get SL sources
```
Find the Sigma connection UUID in **Administration → Connections** or from the `source.connectionId` field in a fetched data model spec. Data model elements whose `connectionId` has no mapping are ingested as wiki-only.
### Workbook filter
At large scale, you can limit which workbooks are fetched during ingest using `workbookFilter`:
```yaml title="ktx.yaml"
connections:
sigma-main:
driver: sigma
client_id: "<your-client-id>"
client_secret_ref: env:SIGMA_CLIENT_SECRET
workbookFilter:
includeArchived: false # default
includeExplorations: false # default
updatedSince: "2026-01-01T00:00:00Z" # only recently updated workbooks
```
| Field | Default | Description |
|-------|---------|-------------|
| `includeArchived` | `false` | Include archived workbooks |
| `includeExplorations` | `false` | Include exploration workbooks |
| `updatedSince` | — | ISO 8601 date; only workbooks updated on or after this date are fetched |
### Notes
- `connectionMappings` is optional for wiki-only ingest; it is required to generate semantic-layer sources and run warehouse validation
- Context ingest (`ktx ingest sigma-main`) fetches from the Sigma API directly
- Ingest is incremental: items whose `updatedAt` timestamp is unchanged since the last run are skipped
- Models backed by CSV uploads or unsupported connector subtypes cannot have their spec exported; these are skipped with a warning (a Sigma API limitation)
- Joins are not projected from Sigma data models in this release; `joins: []` is always written by the projection step. Lookup relationships visible in data model specs are captured as wiki knowledge instead.
---
## Google Drive
Ingests Google Docs from a shared Google Drive folder as wiki-ready knowledge content. This v1 implementation is knowledge-only and ingests Google Docs MIME types only.
### What it provides
- Wiki pages synthesized from Google Docs content
- Folder-scoped knowledge ingestion from a specific Drive folder
- Markdown normalization for headings, lists, paragraphs, links, common inline formatting, and Google Docs tables
### Connection config
```yaml title="ktx.yaml"
connections:
company-docs:
driver: gdrive
service_account_key_ref: file:/absolute/path/to/google-service-account.json
folder_id: your-google-drive-folder-id
recursive: false
```
### Authentication
| Method | Config |
|--------|--------|
| Service account JSON key file | `service_account_key_ref: file:/absolute/path/to/key.json` |
### Google Cloud setup
1. Create a Google Cloud project.
2. Enable the Google Drive API.
3. Enable the Google Docs API.
4. Create a service account.
5. Download the service account JSON key.
6. Share the target Drive folder with the service account email.
7. Reference the key in `ktx.yaml` with `service_account_key_ref`.
### Required scopes
- `https://www.googleapis.com/auth/drive.readonly`
- `https://www.googleapis.com/auth/documents.readonly`
### Configuration options
| Field | Description | Default |
|-------|-------------|---------|
| `service_account_key_ref` | File reference to the service account JSON key | - |
| `folder_id` | Google Drive folder ID to ingest | - |
| `recursive` | Traverse subfolders under `folder_id` | `false` |
### What gets ingested
- Google Docs documents only
- Wiki-oriented knowledge content
- One work unit per staged Google Doc
### Notes
- `gdrive` is knowledge-only in v1; it does not produce semantic layer sources
- `ktx setup` supports Google Drive configuration, including the service-account key ref, folder id, and recursive crawl flag
- `ktx connection test <connectionId>` supports `gdrive`: it verifies that `folder_id` resolves to a folder the service account can read, then reports the number of Google Docs visible in it. A wrong or unshared `folder_id` fails the test instead of reporting zero docs
- Only Google Docs are ingested in v1; other file types (Sheets, Slides, PDFs) in the folder are skipped and recorded in the staged manifest
- The service account must be granted access to the target folder explicitly
## Common errors
| Error or symptom | Likely cause | Recovery |

View file

@ -1,6 +1,6 @@
---
title: Primary Sources
description: Connect ktx to PostgreSQL, Snowflake, BigQuery, MySQL, ClickHouse, SQL Server, SQLite, DuckDB, MongoDB, or Amazon Athena.
description: Connect ktx to PostgreSQL, Snowflake, BigQuery, MySQL, ClickHouse, SQL Server, or SQLite.
---
**ktx** connects to your data warehouse or database to build schema context,
@ -26,23 +26,15 @@ Agents should prefer environment or file references over literal secrets.
| Field | Required | Applies to | Description |
|-------|----------|------------|-------------|
| `driver` | Yes | all connections | Connector driver such as `postgres`, `snowflake`, `bigquery`, `mysql`, `clickhouse`, `sqlserver`, `sqlite`, `duckdb`, `mongodb`, or `athena` |
| `driver` | Yes | all connections | Connector driver such as `postgres`, `snowflake`, `bigquery`, `mysql`, `clickhouse`, `sqlserver`, or `sqlite` |
| `url` | One of the connection methods | URL-style connectors | Database URL, `env:NAME`, or `file:/path/to/secret` |
| `host`, `port`, `database`, `username`, `password` | One of the connection methods | PostgreSQL, MySQL, SQL Server | Field-by-field connection values |
| `schema` or `schemas` | No | schema-aware warehouses | Single schema or list of schemas to scan |
| `databases` | No | ClickHouse, MongoDB, Athena | List of databases to scan |
| `sample_size`, `order_by` | No | MongoDB | Schema-inference sampling controls (recent documents, sort field) |
| `context.queryHistory` | No | PostgreSQL, Snowflake, BigQuery | Enables query-history ingestion when the warehouse supports it |
| `path` | Yes for path-style SQLite/DuckDB | SQLite, DuckDB | Local SQLite or DuckDB database path or `env:NAME` reference |
| `path` | Yes for path-style SQLite | SQLite | Local SQLite database path or `env:NAME` reference |
| `max_bytes_billed` | No | BigQuery | Maximum bytes billed per query job |
| `query_timeout_ms` | No | all warehouses | Maximum execution time for a single read-only query, in milliseconds (default 30000). A query exceeding it is cancelled server-side (or, for SQLite, by terminating the off-process executor) and returns a `query exceeded Ns` error so the agent can revise. |
| `job_timeout_ms` | No | BigQuery | BigQuery query job timeout in milliseconds |
| `project_id` | No | BigQuery | Optional local descriptor and mapping metadata; not used for BigQuery authentication |
| `region` | Yes | Athena | AWS region where the Athena workgroup and Glue catalog reside (e.g. `us-east-1`) |
| `s3_staging_dir` | Yes | Athena | S3 URI for Athena query result storage (e.g. `s3://my-bucket/athena-results/`) |
| `workgroup` | No | Athena | Athena workgroup name (default `primary`) |
| `catalog` | No | Athena | Glue Data Catalog name (default `AwsDataCatalog`) |
| `database` | No | Athena | Default Glue database name passed as the query execution context |
| `databases` | No | Athena | Glue databases to include in schema scans; written by `ktx setup` and read by `ktx ingest` |
## PostgreSQL
@ -226,37 +218,6 @@ BigQuery dataset scope is stored in `connections.<id>.dataset_ids`. Interactive
setup discovers datasets from credentials plus location, then writes the chosen
dataset ids as the scan scope.
### Cross-project datasets
To introspect a dataset hosted in a **different project** than the one your
credentials bill to — for example Google's `bigquery-public-data`, a partner's
shared project, or an organization's central data project — qualify the entry
as `project.dataset`:
```yaml title="ktx.yaml"
connections:
public-bq:
driver: bigquery
credentials_json: file:~/.config/gcloud/bq-service-account.json
location: US
dataset_ids:
- bigquery-public-data.austin_311
- bigquery-public-data.census_bureau_usa
- analytics
```
**ktx** introspects each dataset in its host project while every query job still
bills to the `project_id` inside your `credentials_json`. A bare `dataset` entry
(no prefix) is scanned in your own project, exactly as before. A single
connection may mix datasets from several projects, and two projects may host
datasets with the same name without colliding.
Interactive setup does not enumerate datasets in projects your credentials don't
own, so hand-write `project.dataset` entries for foreign datasets. The wizard's
table picker also only lists datasets in your connection's `location` region;
this affects table selection only — ingest and `discover_data` introspect a
cross-project dataset regardless of region.
### Authentication
| Method | Config |
@ -306,77 +267,7 @@ staged artifact shape as Postgres and Snowflake.
- Parameter binding uses named `@param` syntax
- Arrays flattened to comma-separated strings in results
- Location specified at query execution time
- Supports the `max_bytes_billed` limit from `ktx.yaml`; the shared `query_timeout_ms` field maps to the query job's `jobTimeoutMs`
---
## Amazon Athena
Connects to Amazon Athena using the AWS Glue Data Catalog for schema introspection and the Athena query API for read-only SQL execution. Authentication uses the standard AWS credential chain — no credentials are embedded in `ktx.yaml`.
### Connection config
```yaml title="ktx.yaml"
connections:
my-athena:
driver: athena
region: us-east-1
s3_staging_dir: s3://my-bucket/athena-results/
```
With optional fields:
```yaml title="ktx.yaml"
connections:
my-athena:
driver: athena
region: us-east-1
s3_staging_dir: env:ATHENA_S3_STAGING_DIR
workgroup: analytics
catalog: AwsDataCatalog
database: my_default_database
databases:
- analytics
- raw
```
`ktx setup` writes the `databases` array when you select Glue databases during setup. `ktx scan` reads it to limit introspection to those databases.
### Authentication
**ktx** uses the AWS SDK default credential chain — no credentials appear in `ktx.yaml`. The chain resolves credentials in this order:
| Method | How to configure |
|--------|-----------------|
| Environment variables | Set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and optionally `AWS_SESSION_TOKEN` |
| Shared credentials file | Configure `~/.aws/credentials` with a `[default]` or named profile; set `AWS_PROFILE` to select a non-default profile |
| IAM instance profile | Attach an IAM role to the EC2 instance or ECS task — no local configuration needed |
| IAM roles for service accounts (EKS) | Annotate the pod's service account with the IAM role ARN |
The IAM principal must have `athena:StartQueryExecution`, `athena:GetQueryExecution`, `athena:GetQueryResults`, `glue:GetDatabases`, and `glue:GetTables` permissions, plus read access to the S3 results bucket.
### Features
| Feature | Supported | Notes |
|---------|-----------|-------|
| Tables & views | Yes | Via AWS Glue Data Catalog |
| Primary keys | No | Glue does not expose constraint metadata |
| Foreign keys | No | Not available in Glue/Athena |
| Row count estimates | No | Glue table statistics are often stale |
| Column statistics | No | - |
| Query history | No | - |
| Table sampling | Yes | `SELECT ... LIMIT n` |
### Dialect notes
- SQL dialect is Presto/Trino; identifiers are quoted with double-quotes
- Table names use three-part format: `"catalog"."database"."table"` (e.g. `"AwsDataCatalog"."analytics"."orders"`)
- Partition columns (`PartitionKeys` in Glue) are included after regular columns in the schema and are fully queryable
- Athena does not support `TABLESAMPLE`; random sampling uses `ORDER BY rand()`
- Query execution is asynchronous: **ktx** starts the query, polls until completion, then fetches results from S3
- Results are stored in `s3_staging_dir`; the IAM principal needs write access to that bucket
- Use `workgroup` to apply per-workgroup cost controls and result configuration
- The connector always uses your account's default Glue Data Catalog; cross-account catalog access (`CatalogId` pointing to another account) is not supported
- Supports `max_bytes_billed` and `job_timeout_ms` limits from `ktx.yaml`
---
@ -619,141 +510,12 @@ No authentication required - SQLite is file-based. The file must be readable by
- Foreign key enforcement requires explicit `PRAGMA foreign_keys = ON`
- Database file must exist before `ktx connection test` or ingest runs
---
## DuckDB
File-based connector using the DuckDB Node.js API. Ideal for local analytics, embedded warehouses, and cross-database federation.
### Connection config
```yaml title="ktx.yaml"
connections:
warehouse:
driver: duckdb
path: data/warehouse.duckdb
```
`path` is resolved relative to the project directory. The `.duckdb` file must already exist — **ktx** never creates a missing database file.
### Authentication
No authentication required — DuckDB is file-based. The `.duckdb` file must be readable by the process running **ktx**.
### Features
| Feature | Supported | Notes |
|---------|-----------|-------|
| Tables & views | Yes | Via `information_schema` on the `main` schema |
| Primary keys | Yes | Via `information_schema.table_constraints` |
| Foreign keys | Yes | Via DuckDB's `duckdb_constraints()` catalog function |
| Row count estimates | Yes | Exact count via `SELECT COUNT(*)` |
| Column statistics | No | - |
| Query history | No | - |
| Table sampling | Yes | - |
| Nested analysis | No | - |
### Dialect notes
- Introspection scans the `main` schema only
- Execution is read-only; **ktx** opens the file without write access
- Parameter binding uses positional `?` placeholders
- Uses `LIMIT X OFFSET Y` for pagination
- Database file must exist before `ktx connection test` or ingest runs
### Cross-database federation
When a project declares two or more attach-compatible connections — any combination of `postgres`, `mysql`, `sqlite`, and `duckdb` — **ktx** derives a cross-database federation connection. That connection can ATTACH a native `.duckdb` file, allowing semantic queries to join across sources without manually copying data.
---
## MongoDB
Connects to MongoDB as a primary context source. **ktx** treats each collection
as a table and each inferred top-level field as a column. MongoDB is a non-SQL
source: `ktx sql` and semantic-layer metric compilation do not apply to a MongoDB
connection, but its collections still flow through `ktx ingest`, descriptions, and
relationship discovery.
### Connection config
```yaml title="ktx.yaml"
connections:
mongo-prod:
driver: mongodb
url: env:MONGO_URL
databases: [app]
enabled_tables: [app.users, app.orders] # optional collection allowlist
sample_size: 1000
# order_by: createdAt # only when _id is not an ObjectId
```
Standard `mongodb://` and `mongodb+srv://` connection strings are supported,
including TLS and MongoDB Atlas — pass the full connection string (with its
query parameters) as `url`. The `databases` list selects which databases to
introspect; if omitted, **ktx** uses the database in the URL path. `ktx setup`
also offers MongoDB and stores the selected databases under
`connections.<id>.databases`.
### Authentication
| Method | Config |
|--------|--------|
| Connection string | `url: env:MONGO_URL` or `url: file:/path/to/secret` |
| Atlas / TLS | Use a `mongodb+srv://` URL with the credentials and TLS options Atlas provides |
### Schema inference
MongoDB has no fixed schema, so **ktx** infers one by sampling the most recent
`sample_size` documents per collection (default 1000), sorted by `_id`
descending. Because an ObjectId embeds its creation time, this captures the
collection's current shape with zero configuration. When `_id` is not an
ObjectId (custom string or UUID keys), set `order_by` to a timestamp field such
as `createdAt` so "most recent" is well-defined. A custom `order_by` field
should be indexed — an unindexed sort hits MongoDB's in-memory sort limit and
fails on large collections (`_id`, the default, is always indexed).
For each top-level field, **ktx** unions the BSON types seen and derives
nullability from how often the field is present:
- Scalar BSON types map to `string`, `number`, `time`, or `boolean`
- A field seen with more than one type is recorded as `mixed` and treated as a string
- Sub-documents and arrays become a single opaque `json` column (no dotted-path
columns); their sampled values are stringified, not faithfully serialized
- `_id` is the primary key
### Features
| Feature | Supported | Notes |
|---------|-----------|-------|
| Collections (as tables) | Yes | Via `listCollections`; `system.*` collections are excluded |
| Primary keys | Yes | `_id` |
| Foreign keys | No | MongoDB has no formal foreign keys |
| Row count estimates | Yes | Via `estimatedDocumentCount` |
| Column statistics | No | - |
| Query history | No | - |
| Table sampling | Yes | Reads the most recent documents |
| Nested analysis | Yes | Sub-documents and arrays modeled as opaque `json` |
| Read-only SQL (`ktx sql`) | No | MongoDB is not a SQL source |
### Dialect notes
- Strictly read-only: the connector only issues `find`, `listCollections`,
`estimatedDocumentCount`, and read aggregations
- Sampling rides the `_id` index and uses a server-side time limit so large
collections do not stall a run; a custom `order_by` must be indexed for the
same guarantee
- `sample_size` trades inference coverage for speed; raise it for collections
with highly variable documents
## Common errors
| Error or symptom | Likely cause | Recovery |
|------------------|--------------|----------|
| Connection URL appears in git diff | A literal credential URL was written to `ktx.yaml` | Replace it with `env:NAME` or `file:/path/to/secret` and rotate exposed credentials |
| Database ingest returns no tables | Schema, database, or project filter is wrong, or the user lacks metadata permissions | Verify the schema list and grant metadata read permissions. For Athena, confirm the IAM principal has `glue:GetDatabases` and `glue:GetTables` permissions |
| Database ingest returns no tables | Schema, database, or project filter is wrong, or the user lacks metadata permissions | Verify the schema list and grant metadata read permissions |
| Query history is empty | Query history extension or warehouse history view is unavailable | Enable the warehouse-specific history feature, then rerun `ktx ingest <connectionId> --query-history` or `ktx setup` |
| Column statistics are missing | Connector cannot access stats tables or the warehouse does not expose them | Grant stats permissions where supported; otherwise rely on schema-level context without column statistics |
| Column statistics are missing | Connector cannot access stats tables or the warehouse does not expose them | Grant stats permissions where supported; otherwise rely on fast schema context |
| Semantic query execution fails | Connection is missing, unreachable, or query execution is disabled | Run `ktx connection test <id>` and check the `ktx sl query` flags |
| Athena query fails with `ACCESS_DENIED` | IAM principal lacks `athena:StartQueryExecution` or S3 write access to `s3_staging_dir` | Attach a policy granting Athena query permissions and `s3:PutObject` on the staging bucket |
| Athena ingest finds databases but no tables | IAM principal has `glue:GetDatabases` but not `glue:GetTables` | Grant `glue:GetTables` on the relevant Glue catalog resources |

View file

@ -8,6 +8,7 @@
"integrations",
"configuration",
"cli-reference",
"ai-resources",
"community"
]
}

View file

@ -54,7 +54,9 @@ ktx provides semantic-layer files, warehouse scans, wiki pages, provenance, and
- Installable setup skill: run \`npx skills add Kaelio/ktx --skill ktx\` from
the project you want to configure.
${link("/docs/community/ai-resources", "AI Resources", "How coding agents read, cite, and act on the ktx docs")}
${link("/docs/ai-resources/agent-quickstart", "Agent Quickstart", "Task-first route for coding assistants using ktx")}
${link("/docs/ai-resources/markdown-access", "Markdown Access", "Fetch ktx docs as llms.txt, llms-full.txt, or per-page Markdown")}
${link("/docs/ai-resources/agent-instructions", "Agent Instructions", "Suggested instructions for coding assistants that need to read and cite ktx docs")}
## Start Here
@ -65,7 +67,7 @@ ${link("/docs/guides/writing-context", "Writing Context", "Write semantic source
## Machine-Readable Documentation
- [Full documentation](${absoluteUrl("/llms-full.txt")}): All docs pages in one plain-text markdown response
- [AI Resources guide](${absoluteUrl("/docs/community/ai-resources.md")}): How agents fetch llms.txt, llms-full.txt, and per-page Markdown
- [Markdown access guide](${absoluteUrl("/docs/ai-resources/markdown-access.md")}): How to fetch llms.txt, llms-full.txt, and per-page Markdown
- [Quickstart markdown](${absoluteUrl("/docs/getting-started/quickstart.md")}): Human setup walkthrough
- [Semantic-layer CLI markdown](${absoluteUrl("/docs/cli-reference/ktx-sl.md")}): Semantic-layer commands and JSON output
- [Wiki CLI markdown](${absoluteUrl("/docs/cli-reference/ktx-wiki.md")}): Wiki page commands and JSON output
@ -145,8 +147,8 @@ function absoluteUrl(path: string) {
function formatCategoryName(category: string) {
const labels: Record<string, string> = {
"ai-resources": "AI Resources",
"cli-reference": "CLI Reference",
community: "Community & Resources",
};
if (labels[category]) {

View file

@ -30,36 +30,7 @@ const config = {
};
},
async redirects() {
// Alias-host canonicalization MUST come before the generic root/docs
// redirects below. Those generic rules have no host guard, so if they ran
// first they would inject a "/ktx" basePath into the path on the alias
// hosts, which the alias catch-alls would then prepend a second time —
// producing https://docs.kaelio.com/ktx/ktx/docs/... Redirects also run
// before beforeFiles rewrites, so the ktx.sh catch-all must exclude
// /stars* to let the stars dashboard rewrite proxy through.
return [
{
source: "/slack",
has: [{ type: "host", value: "ktx.sh" }],
destination:
"https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ",
permanent: false,
basePath: false,
},
{
source: "/:path*",
has: [{ type: "host", value: "docs.ktx.sh" }],
destination: "https://docs.kaelio.com/ktx/:path*",
permanent: true,
basePath: false,
},
{
source: "/:path((?!stars(?:/|$)).*)",
has: [{ type: "host", value: "ktx.sh" }],
destination: "https://docs.kaelio.com/ktx/:path",
permanent: true,
basePath: false,
},
{
source: "/",
destination: "/ktx/docs/getting-started/introduction",
@ -73,30 +44,26 @@ const config = {
basePath: false,
},
{
// AI Resources collapsed from four pages to one and now lives under the
// Community & Resources section. Redirect the old top-level URL and the
// retired per-page slugs to the new home. Redirects run before the .md
// rewrite, so the Markdown variants must be matched first and keep their
// .md suffix; otherwise a cached Markdown URL would 308 to the HTML page
// and break the agent Markdown contract.
source: "/docs/ai-resources.md",
destination: "/docs/community/ai-resources.md",
source: "/:path*",
has: [{ type: "host", value: "docs.ktx.sh" }],
destination: "https://docs.kaelio.com/ktx/:path*",
permanent: true,
basePath: false,
},
{
source: "/docs/ai-resources/:slug([^/]+\\.md)",
destination: "/docs/community/ai-resources.md",
permanent: true,
source: "/slack",
has: [{ type: "host", value: "ktx.sh" }],
destination:
"https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ",
permanent: false,
basePath: false,
},
{
source: "/docs/ai-resources",
destination: "/docs/community/ai-resources",
permanent: true,
},
{
source: "/docs/ai-resources/:slug",
destination: "/docs/community/ai-resources",
source: "/:path((?!stars(?:/|$)).*)",
has: [{ type: "host", value: "ktx.sh" }],
destination: "https://docs.kaelio.com/ktx/:path",
permanent: true,
basePath: false,
},
];
},

View file

@ -12,16 +12,15 @@
"dependencies": {
"@xyflow/react": "^12.10.2",
"fumadocs-core": "16.8.10",
"fumadocs-mdx": "15.0.7",
"fumadocs-mdx": "15.0.4",
"fumadocs-ui": "16.8.10",
"html-to-image": "1.11.11",
"next": "^16",
"react": "19.2.6",
"react-dom": "19.2.6"
},
"devDependencies": {
"@tailwindcss/postcss": "^4",
"@types/node": "^25.9.1",
"@types/node": "^25.7.0",
"@types/react": "^19",
"@types/react-dom": "^19",
"tailwindcss": "^4",

View file

@ -0,0 +1,210 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1346" height="1710" viewBox="0 0 1346 1710" role="img" aria-labelledby="title desc">
<title id="title">ktx ingestion flow</title>
<desc id="desc">Source systems flow through source connectors, context builder, reconciliation, and validation to create wiki Markdown and semantic-layer YAML outputs.</desc>
<defs>
<filter id="card-shadow" x="-12%" y="-12%" width="124%" height="124%" color-interpolation-filters="sRGB">
<feDropShadow dx="0" dy="2" stdDeviation="2" flood-color="#0f172a" flood-opacity="0.14"/>
</filter>
<filter id="dark-shadow" x="-12%" y="-12%" width="124%" height="124%" color-interpolation-filters="sRGB">
<feDropShadow dx="0" dy="2" stdDeviation="2" flood-color="#020617" flood-opacity="0.22"/>
</filter>
<filter id="glow-blue" x="-160%" y="-160%" width="420%" height="420%">
<feGaussianBlur stdDeviation="7" result="blur"/>
<feMerge>
<feMergeNode in="blur"/>
<feMergeNode in="SourceGraphic"/>
</feMerge>
</filter>
<marker id="arrow" viewBox="0 0 10 10" refX="8.5" refY="5" markerWidth="9" markerHeight="9" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#94a3b8"/>
</marker>
<style>
.card { fill: #ffffff; stroke: #e2e8f0; stroke-width: 1.4; filter: url(#card-shadow); }
.stage { fill: #0b1f23; stroke: #17343a; stroke-width: 1.2; filter: url(#dark-shadow); }
.title { fill: #24272d; font: 700 28px Inter, Arial, sans-serif; }
.body { fill: #666b73; font: 500 18px Inter, Arial, sans-serif; }
.tag { fill: #6b7280; font: 500 16px Inter, Arial, sans-serif; }
.mono { font: 700 20px "SFMono-Regular", Consolas, monospace; }
.stage-title { fill: #f8fafc; font: 700 28px Inter, Arial, sans-serif; }
.stage-body { fill: #b8c6ca; font: 500 20px Inter, Arial, sans-serif; }
.index { fill: #07313a; font: 700 22px Inter, Arial, sans-serif; text-anchor: middle; dominant-baseline: middle; }
.edge { fill: none; stroke: #94a3b8; stroke-width: 2; stroke-linecap: round; stroke-linejoin: round; }
.dash { fill: none; stroke: #64748b; stroke-width: 1.8; stroke-dasharray: 5 8; stroke-linecap: round; }
</style>
</defs>
<g id="source-cards">
<g transform="translate(24 39)">
<rect class="card" x="0" y="0" width="298" height="285" rx="4"/>
<rect x="0" y="0" width="298" height="4" rx="2" fill="#3b82f6"/>
<text class="title" x="22" y="52">Databases</text>
<text class="body" x="22" y="92">Schemas, columns, keys,</text>
<text class="body" x="22" y="120">row counts, and query</text>
<text class="body" x="22" y="148">history.</text>
<g transform="translate(22 180)">
<rect x="0" y="0" width="112" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">PostgreSQL</text>
<rect x="120" y="0" width="100" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="132" y="24">Snowflake</text>
<rect x="0" y="46" width="92" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="70">BigQuery</text>
<rect x="100" y="46" width="74" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="112" y="70">SQLite</text>
</g>
</g>
<g transform="translate(358 39)">
<rect class="card" x="0" y="0" width="298" height="285" rx="4"/>
<rect x="0" y="0" width="298" height="4" rx="2" fill="#f97316"/>
<text class="title" x="22" y="52">BI tools</text>
<text class="body" x="22" y="92">Dashboards, questions,</text>
<text class="body" x="22" y="120">explores, usage, and trusted</text>
<text class="body" x="22" y="148">examples.</text>
<g transform="translate(22 180)">
<rect x="0" y="0" width="96" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">Metabase</text>
<rect x="104" y="0" width="74" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="116" y="24">Looker</text>
</g>
</g>
<g transform="translate(692 39)">
<rect class="card" x="0" y="0" width="298" height="285" rx="4"/>
<rect x="0" y="0" width="298" height="4" rx="2" fill="#f59e0b"/>
<text class="title" x="22" y="52">Modeling code</text>
<text class="body" x="22" y="92">Existing metrics, dimensions,</text>
<text class="body" x="22" y="120">models, joins, and entities.</text>
<g transform="translate(22 152)">
<rect x="0" y="0" width="48" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">dbt</text>
<rect x="56" y="0" width="82" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="68" y="24">LookML</text>
<rect x="0" y="46" width="102" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="70">MetricFlow</text>
</g>
</g>
<g transform="translate(1026 39)">
<rect class="card" x="0" y="0" width="298" height="285" rx="4"/>
<rect x="0" y="0" width="298" height="4" rx="2" fill="#10b981"/>
<text class="title" x="22" y="52">Docs and notes</text>
<text class="body" x="22" y="92">Policies, caveats, team</text>
<text class="body" x="22" y="120">definitions, and analyst</text>
<text class="body" x="22" y="148">context.</text>
<g transform="translate(22 180)">
<rect x="0" y="0" width="72" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">Notion</text>
<rect x="80" y="0" width="84" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="92" y="24">Any text</text>
</g>
</g>
</g>
<g id="edges">
<path class="edge" d="M172 324 V380 Q172 394 186 394 H507 Q507 394 507 380 V324"/>
<path class="edge" d="M841 324 V380 Q841 394 827 394 H507"/>
<path class="edge" d="M1175 324 V380 Q1175 394 1161 394 H673 Q673 394 673 408 V433" marker-end="url(#arrow)"/>
<path class="edge" d="M507 394 H673"/>
<path class="edge" d="M673 618 V651" marker-end="url(#arrow)"/>
<path class="edge" d="M673 833 V866" marker-end="url(#arrow)"/>
<path class="edge" d="M673 1048 V1081" marker-end="url(#arrow)"/>
<path class="edge" d="M673 1262 V1310 Q673 1325 656 1325 H305 Q291 1325 291 1339 V1364" marker-end="url(#arrow)"/>
<path class="edge" d="M673 1262 V1310 Q673 1325 690 1325 H1043 Q1057 1325 1057 1339 V1364" marker-end="url(#arrow)"/>
<path class="dash" d="M546 1523 H800"/>
<path d="M546 1523 l9 -6 v12 z" fill="#64748b"/>
<path d="M800 1523 l-9 -6 v12 z" fill="#64748b"/>
</g>
<g id="particles">
<circle cx="256" cy="394" r="18" fill="#3b82f6" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="256" cy="394" r="6" fill="#3b82f6" opacity="0.9"/>
<circle cx="632" cy="394" r="18" fill="#f97316" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="632" cy="394" r="6" fill="#f97316" opacity="0.9"/>
<circle cx="830" cy="394" r="18" fill="#10b981" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="830" cy="394" r="6" fill="#10b981" opacity="0.9"/>
<circle cx="673" cy="635" r="17" fill="#10b981" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="673" cy="635" r="6" fill="#10b981" opacity="0.9"/>
<circle cx="673" cy="1065" r="17" fill="#f59e0b" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="673" cy="1065" r="6" fill="#f59e0b" opacity="0.9"/>
<circle cx="573" cy="1322" r="17" fill="#3b82f6" opacity="0.18" filter="url(#glow-blue)"/>
<circle cx="573" cy="1322" r="6" fill="#3b82f6" opacity="0.9"/>
</g>
<g id="stages">
<g transform="translate(464 438)">
<rect class="stage" x="0" y="0" width="420" height="180" rx="4"/>
<circle cx="52" cy="90" r="26" fill="#55dced"/>
<text class="index" x="52" y="90">1</text>
<text class="stage-title" x="98" y="72">Source connectors</text>
<text class="stage-body" x="98" y="110">Read each configured system in</text>
<text class="stage-body" x="98" y="140">its native shape.</text>
</g>
<g transform="translate(464 653)">
<rect class="stage" x="0" y="0" width="420" height="180" rx="4"/>
<circle cx="52" cy="90" r="26" fill="#55dced"/>
<text class="index" x="52" y="90">2</text>
<text class="stage-title" x="98" y="72">Context builder</text>
<text class="stage-body" x="98" y="110">Turn source evidence into</text>
<text class="stage-body" x="98" y="140">proposed context updates.</text>
</g>
<g transform="translate(464 868)">
<rect class="stage" x="0" y="0" width="420" height="180" rx="4"/>
<circle cx="52" cy="90" r="26" fill="#55dced"/>
<text class="index" x="52" y="90">3</text>
<text class="stage-title" x="98" y="72">Reconciliation</text>
<text class="stage-body" x="98" y="110">Merge new evidence with the</text>
<text class="stage-body" x="98" y="140">context that already exists.</text>
</g>
<g transform="translate(464 1082)">
<rect class="stage" x="0" y="0" width="420" height="180" rx="4"/>
<circle cx="52" cy="90" r="26" fill="#55dced"/>
<text class="index" x="52" y="90">4</text>
<text class="stage-title" x="98" y="72">Validation</text>
<text class="stage-body" x="98" y="110">Check references and semantics</text>
<text class="stage-body" x="98" y="140">before agents rely on them.</text>
</g>
</g>
<g id="outputs">
<g transform="translate(60 1373)">
<rect class="card" x="0" y="0" width="485" height="329" rx="4"/>
<rect x="0" y="0" width="485" height="4" rx="2" fill="#10b981"/>
<text class="mono" x="24" y="52" fill="#10b981">wiki/*.md</text>
<text class="title" x="24" y="100">Wiki</text>
<g transform="translate(24 122)">
<rect x="0" y="0" width="90" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">free-form</text>
<rect x="98" y="0" width="140" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="110" y="24">auto-maintained</text>
</g>
<text class="body" x="24" y="194">Definitions, caveats, policies, analyst notes, and</text>
<text class="body" x="24" y="222">business language that agents can search.</text>
</g>
<g transform="translate(803 1373)">
<rect class="card" x="0" y="0" width="485" height="329" rx="4"/>
<rect x="0" y="0" width="485" height="4" rx="2" fill="#3b82f6"/>
<text class="mono" x="24" y="52" fill="#3b82f6">semantic-layer/*.yaml</text>
<text class="title" x="24" y="100">Semantic layer</text>
<g transform="translate(24 122)">
<rect x="0" y="0" width="96" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="12" y="24">structured</text>
<rect x="104" y="0" width="104" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="116" y="24">executable</text>
<rect x="216" y="0" width="140" height="36" rx="4" fill="#fbfaf8" stroke="#e5e1dc"/>
<text class="tag" x="228" y="24">auto-maintained</text>
</g>
<text class="body" x="24" y="194">Metrics, joins, tables, dimensions, filters, and</text>
<text class="body" x="24" y="222">segments that ktx can validate and compile into</text>
<text class="body" x="24" y="250">SQL.</text>
</g>
<g transform="translate(622 1505)">
<rect x="0" y="0" width="102" height="36" rx="4" fill="#ffffff" stroke="#e5e1dc"/>
<text class="tag" x="13" y="24">references</text>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 346 KiB

After

Width:  |  Height:  |  Size: 137 KiB

Before After
Before After

Binary file not shown.

Before

Width:  |  Height:  |  Size: 176 KiB

View file

@ -2,8 +2,6 @@ import assert from "node:assert/strict";
import { spawn } from "node:child_process";
import { once } from "node:events";
import { readFile, writeFile } from "node:fs/promises";
import http from "node:http";
import https from "node:https";
import { dirname, join } from "node:path";
import { createServer } from "node:net";
import { after, before, test } from "node:test";
@ -102,37 +100,6 @@ after(async () => {
}
});
// Node's fetch (undici) overwrites the Host header with the connection host,
// so the alias-host redirect rules never match. The low-level http(s) client
// sends Host verbatim, which is what the alias canonicalization keys off of.
function requestWithHost(hostHeader, path) {
const target = new URL(docsSiteUrl);
const client = target.protocol === "https:" ? https : http;
const port =
target.port || (target.protocol === "https:" ? "443" : "80");
return new Promise((resolve, reject) => {
const request = client.request(
{
hostname: target.hostname,
port,
path,
method: "GET",
headers: { Host: hostHeader },
},
(response) => {
response.resume();
resolve({
status: response.statusCode,
location: response.headers.location,
});
},
);
request.on("error", reject);
request.end();
});
}
test("/ktx/docs redirects to the docs introduction", async () => {
const response = await fetch(`${docsSiteUrl}${docsBasePath}/docs`, {
redirect: "manual",
@ -145,53 +112,6 @@ test("/ktx/docs redirects to the docs introduction", async () => {
);
});
test("retired AI Resources URLs redirect to the page under Community", async () => {
// The former top-level URL.
const bare = await fetch(
`${docsSiteUrl}${docsBasePath}/docs/ai-resources`,
{ redirect: "manual" },
);
assert.equal(bare.status, 308);
assert.equal(
bare.headers.get("location"),
`${docsBasePath}/docs/community/ai-resources`,
);
// A retired per-page slug.
const slug = await fetch(
`${docsSiteUrl}${docsBasePath}/docs/ai-resources/agent-quickstart`,
{ redirect: "manual" },
);
assert.equal(slug.status, 308);
assert.equal(
slug.headers.get("location"),
`${docsBasePath}/docs/community/ai-resources`,
);
// A retired per-page Markdown URL must stay Markdown: it has to redirect to
// the new .md route, not fall through to the HTML page.
const markdown = await fetch(
`${docsSiteUrl}${docsBasePath}/docs/ai-resources/agent-quickstart.md`,
{ redirect: "manual" },
);
assert.equal(markdown.status, 308);
assert.equal(
markdown.headers.get("location"),
`${docsBasePath}/docs/community/ai-resources.md`,
);
// Following that redirect end to end must land on Markdown, not HTML.
const followed = await fetch(
`${docsSiteUrl}${docsBasePath}/docs/ai-resources/agent-quickstart.md`,
);
assert.equal(followed.status, 200);
assert.match(followed.headers.get("content-type") ?? "", /text\/markdown/);
});
test("/ redirects into the /ktx docs site", async () => {
const response = await fetch(`${docsSiteUrl}/`, {
redirect: "manual",
@ -221,51 +141,3 @@ test("/ktx/api/search returns docs search results", async () => {
"search should return at least one docs result",
);
});
test("ktx.sh canonicalizes to a single /ktx basePath on the docs host", async () => {
const root = await requestWithHost("ktx.sh", "/");
assert.equal(root.status, 308);
assert.equal(root.location, "https://docs.kaelio.com/ktx/");
assert.ok(
!root.location.includes("/ktx/ktx"),
"the basePath must not be doubled",
);
const page = await requestWithHost(
"ktx.sh",
"/docs/getting-started/quickstart",
);
assert.equal(page.status, 308);
assert.equal(
page.location,
"https://docs.kaelio.com/ktx/docs/getting-started/quickstart",
);
});
test("docs.ktx.sh canonicalizes to a single /ktx basePath on the docs host", async () => {
const root = await requestWithHost("docs.ktx.sh", "/");
assert.equal(root.status, 308);
assert.equal(root.location, "https://docs.kaelio.com/ktx");
assert.ok(
!root.location.includes("/ktx/ktx"),
"the basePath must not be doubled",
);
const page = await requestWithHost("docs.ktx.sh", "/llms.txt");
assert.equal(page.status, 308);
assert.equal(page.location, "https://docs.kaelio.com/ktx/llms.txt");
});
test("ktx.sh keeps the /slack and /stars exceptions", async () => {
const slack = await requestWithHost("ktx.sh", "/slack");
assert.equal(slack.status, 307);
assert.match(slack.location, /^https:\/\/join\.slack\.com\//);
// /stars is proxied by a beforeFiles rewrite, so the apex catch-all must not
// canonicalize it to the docs host.
const stars = await requestWithHost("ktx.sh", "/stars");
assert.ok(
!(stars.location ?? "").startsWith("https://docs.kaelio.com"),
"the stars dashboard must not be redirected to the docs host",
);
});

View file

@ -85,7 +85,7 @@ test("product mechanics component explains ingestion outputs", async () => {
"compile into SQL",
'"use client"',
"@xyflow/react",
"<FlowCanvas",
"<ReactFlow",
"getSmoothStepPath",
"animateMotion",
"mechanics-particle",
@ -97,21 +97,21 @@ test("product mechanics component explains ingestion outputs", async () => {
);
}
// The ReactFlow canvas config lives in the shared FlowCanvas wrapper, which
// product-mechanics renders. Assert the static read-only behavior there.
const flowCanvas = await readDocsFile("components/flow-canvas.tsx");
for (const guard of [
assert.match(
component,
/nodesDraggable=\{false\}/,
/nodesConnectable=\{false\}/,
"ReactFlow canvas should disable node dragging",
);
assert.match(
component,
/panOnDrag=\{false\}/,
"ReactFlow canvas should disable panning",
);
assert.match(
component,
/zoomOnScroll=\{false\}/,
/elementsSelectable=\{false\}/,
]) {
assert.match(
flowCanvas,
guard,
`shared FlowCanvas should enforce static read-only behavior: ${guard}`,
);
}
"ReactFlow canvas should disable scroll zoom",
);
assert.doesNotMatch(component, /raw-sources/);
assert.doesNotMatch(component, /\.ktx/);

View file

@ -1,74 +0,0 @@
import assert from "node:assert/strict";
import { readFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { test } from "node:test";
import { fileURLToPath } from "node:url";
const docsSiteDir = join(dirname(fileURLToPath(import.meta.url)), "..");
async function readDocsFile(path) {
return readFile(join(docsSiteDir, path), "utf8");
}
test("docs introduction renders the serving phase after ingestion", async () => {
const introduction = await readDocsFile(
"content/docs/getting-started/introduction.mdx",
);
assert.match(
introduction,
/import\s+\{\s*ProductRuntime\s*\}\s+from\s+"@\/components\/product-runtime";/,
);
assert.match(introduction, /<ProductRuntime\s*\/>/);
const mechanicsIndex = introduction.indexOf("<ProductMechanics />");
const runtimeIndex = introduction.indexOf("<ProductRuntime />");
const useCaseIndex = introduction.indexOf("## Use it for");
assert.ok(
runtimeIndex > mechanicsIndex,
"serving diagram should appear after the ingestion diagram",
);
assert.ok(
runtimeIndex < useCaseIndex,
"serving diagram should appear before use-case sections",
);
});
test("product runtime component explains the serving cycle", async () => {
const component = await readDocsFile("components/product-runtime.tsx");
for (const expectedText of [
"How serving works",
"Serving flow",
"From an agent request to a governed answer",
"Your agent",
"Claude Code",
"Cursor",
"Codex",
"Search wiki + semantic layer",
"Return approved metrics",
"Compile metrics → SQL",
"Context layer",
"Database",
"search + read",
"read-only",
"wiki/*.md",
"semantic-layer/*.yaml",
'"use client"',
"@xyflow/react",
"FlowCanvas",
"getSmoothStepPath",
"animateMotion",
"runtime-particle",
"buildCyclePath",
]) {
assert.ok(
component.includes(expectedText),
`component should include: ${expectedText}`,
);
}
assert.doesNotMatch(component, /raw-sources/);
assert.doesNotMatch(component, /<img/);
});

View file

@ -1,4 +1,4 @@
# ktx release runbook
# KTX release runbook
This runbook covers the maintainer workflow for publishing `@kaelio/ktx` to
npm through GitHub Actions. The workflow uses semantic-release to choose the
@ -36,7 +36,7 @@ Before you publish, confirm these requirements:
publish the first stable version as `0.1.0`.
semantic-release doesn't support choosing an arbitrary first `0.x` stable
release. If **ktx** has no stable tag yet and you need the first stable release to
release. If KTX has no stable tag yet and you need the first stable release to
be `0.1.0`, create and push the baseline tag once before running the live
stable workflow:
@ -46,7 +46,7 @@ git tag v0.0.0 "${root_commit}"
git push origin v0.0.0
```
**ktx** follows the same versioning schema as the main Kaelio release workflow:
KTX follows the same versioning schema as the main Kaelio release workflow:
breaking-change and `major` commit markers create a minor release, not an
automatic major release. A major version requires an intentional manual release
path.

View file

@ -77,6 +77,8 @@ maintains, validates, and serves that layer.
| Connection ref in prose | **connection id** (lowercase, two words) | "connection ID" |
| CLI arg/flag literal | `connectionId` (code font) | — |
| File path placeholder | `<connection-id>` (code font) | — |
| Fast schema mode | **fast ingest** | schema ingest, schema-only ingest |
| AI-enriched mode | **deep ingest** | AI-enriched ingest |
| Ingest of a primary connection | **database ingest** | — |
| Ingest of a context-source connection | **context-source ingest** | bare "source ingest" |
| Wiki capture | **text ingest** | — |

View file

@ -17,25 +17,17 @@ The copied project initializes its own Git repository on first use.
## orbit-relationship-verification
`orbit-relationship-verification/` is a checked-in **ktx** project used by
`orbit-relationship-verification/` is a checked-in KTX project used by
`pnpm run relationships:verify-orbit`. It points the `orbit` SQLite connection
at the Orbit-style no-declared-constraint relationship fixture and verifies that
relationship enrichment writes nine accepted joins without requiring a local
warehouse credential.
## mongodb
`mongodb/` is a manual container-backed example for the MongoDB connector. It
seeds a representative dataset (nested documents, arrays, a mixed-type field, a
nullable field, and a view), then exercises the connector as a fast no-LLM
introspection smoke (`scripts/smoke.sh`) and documents a full keyless
`ktx ingest` run. Works with Docker Compose or `podman compose`.
## postgres-historic
`postgres-historic/` is a manual Docker-backed smoke for Postgres
query-history ingest via `pg_stat_statements`. It verifies setup, staged
query-history artifacts, **ktx** daemon batch SQL analysis, bounded pattern
query-history artifacts, KTX daemon batch SQL analysis, bounded pattern
WorkUnit shards, and no-WorkUnit idempotency for unchanged bucketed table
inputs and pattern shards.

View file

@ -1,6 +1,6 @@
# local-warehouse fixture
This directory is a contributor fixture for **ktx** CLI smoke tests. It uses the
This directory is a contributor fixture for KTX CLI smoke tests. It uses the
internal fake ingest adapter so tests can run without a live database or
external service.

View file

@ -5,6 +5,7 @@ storage:
state: sqlite
search: sqlite-fts5
git:
auto_commit: true
author: "ktx <ktx@example.com>"
ingest:
adapters:
@ -17,3 +18,5 @@ agent:
- sl_query
- wiki_search
- sl_read_source
memory:
auto_commit: true

View file

@ -1,127 +0,0 @@
# MongoDB Connector Example
A manual, self-contained example for the **ktx** MongoDB connector. It starts a
local MongoDB, seeds a representative dataset, and exercises the connector both
as a fast no-LLM introspection smoke and as a full `ktx ingest` run.
MongoDB is a **context-only** primary source: collections become tables and
inferred top-level fields become columns, but `ktx sql` and semantic-layer
metric compilation do not apply. See
[`docs-site/content/docs/integrations/primary-sources.mdx`](../../docs-site/content/docs/integrations/primary-sources.mdx).
## Prerequisites
- Docker with Compose v2, or Podman with `podman compose`
- Node and pnpm matching the **ktx** workspace
- The built CLI: `pnpm --filter @kaelio/ktx run build`
- For the full ingest only: `uv` on `PATH` and a usable local Claude Code
session (the keyless `claude-code` LLM backend)
## What the seed contains
[`init/seed.js`](init/seed.js) creates the `app` database with:
- `users``_id` (ObjectId), scalar fields, a nested `address`, an array
`tags`, a `Decimal128` `balance`, a `ref` field that holds more than one type
(inferred `mixed`), and an `age` field absent from one document (nullable)
- `orders` — an ObjectId `user_id` reference for relationship discovery
- `active_users` — a **view** (to confirm introspection never runs a count
command on a view)
MongoDB applies the script once on first container start. Apply it by hand with:
```bash
mongosh "mongodb://localhost:27117" < examples/mongodb/init/seed.js
```
## Smoke (no LLM credentials)
From the **ktx** repository root:
```bash
examples/mongodb/scripts/smoke.sh
```
It starts MongoDB on `127.0.0.1:27117`, seeds it, and asserts the connector's
inferred schema (collections → tables, nested → `json`, `mixed`, nullability,
`_id` primary key, and a view introspected with `estimatedRows: null`). This
drives the same entry point `ktx ingest`'s "database schema" stage uses, without
needing an LLM or embeddings.
Podman:
```bash
KTX_MONGODB_COMPOSE="podman compose" examples/mongodb/scripts/smoke.sh
```
Set `KTX_MONGODB_KEEP=1` to leave the container running after the script exits.
## Full `ktx ingest`
The public database-ingest path requires a configured model and embeddings.
This runs entirely locally with the keyless `claude-code` LLM backend and the
**ktx**-managed `sentence-transformers` embedding daemon — no API keys.
Start MongoDB and create a project:
```bash
docker compose -f examples/mongodb/docker-compose.yml up -d --wait # or: podman compose
node packages/cli/dist/bin.js admin init /tmp/ktx-mongodb-example
```
Add the connection and a keyless enrichment stack to
`/tmp/ktx-mongodb-example/ktx.yaml`:
```yaml
connections:
mongo-prod:
driver: mongodb
url: mongodb://localhost:27117/app
databases:
- app
llm:
provider:
backend: claude-code
models:
default: sonnet
scan:
enrichment:
mode: llm
embeddings:
backend: sentence-transformers
model: all-MiniLM-L6-v2
dimensions: 384
sentenceTransformers:
base_url: ""
```
Test the connection and ingest:
```bash
node packages/cli/dist/bin.js connection test mongo-prod --project-dir /tmp/ktx-mongodb-example
node packages/cli/dist/bin.js ingest mongo-prod --project-dir /tmp/ktx-mongodb-example --yes --plain
```
The first ingest starts the **ktx** embedding daemon and downloads the
`all-MiniLM-L6-v2` model. Expected final state: `Database schema: done`.
Inspect the result:
- `raw-sources/mongo-prod/live-database/<run>/tables/*.json` — one per
collection, including the `active_users` view with `estimatedRows: null`
- `raw-sources/mongo-prod/live-database/<run>/enrichment/relationships.json`
inferred relationships sit in `review` (a non-SQL source has no read-only SQL
coverage validation), with `accepted: []`
- `semantic-layer/mongo-prod/_schema/app.yaml` — the schema with per-column AI
descriptions
`ktx sql -c mongo-prod "SELECT 1"` is refused by the read-only SQL capability
gate, and `ktx sl query -c mongo-prod ...` is refused because MongoDB is not a
SQL source.
## Cleanup
```bash
docker compose -f examples/mongodb/docker-compose.yml down -v # or: podman compose
rm -rf /tmp/ktx-mongodb-example
```

View file

@ -1,14 +0,0 @@
services:
mongodb:
image: mongo:7
ports:
# Non-default host port so the example does not clash with a local MongoDB.
- "27117:27017"
healthcheck:
test: ["CMD-SHELL", "mongosh --quiet --eval \"db.runCommand({ ping: 1 }).ok\" | grep -q 1"]
interval: 2s
timeout: 5s
retries: 30
volumes:
# MongoDB runs *.js here once, on first start, against an empty data dir.
- ./init:/docker-entrypoint-initdb.d:ro

View file

@ -1,64 +0,0 @@
// Seed a representative MongoDB dataset for the ktx connector example.
//
// MongoDB runs this once on first container start (it is mounted into
// /docker-entrypoint-initdb.d). It can also be applied by hand:
// mongosh "mongodb://localhost:27117" < examples/mongodb/init/seed.js
//
// The shapes here exercise the connector's schema inference end to end:
// scalar BSON types, a nested sub-document, an array, Decimal128, dates, a
// field with more than one type (-> "mixed"), an absent field (-> nullable),
// an ObjectId reference for relationship discovery, and a view (to confirm
// introspection never runs a count command on a view).
const app = db.getSiblingDB('app');
app.users.drop();
app.orders.drop();
app.users.insertMany([
{
email: 'ada@example.com',
age: 31,
active: true,
created: new Date('2026-01-04T10:00:00Z'),
balance: NumberDecimal('120.50'),
address: { city: 'NY', zip: '10001' },
tags: ['admin', 'early-access'],
ref: 'abc',
},
{
email: 'grace@example.com',
active: false,
created: new Date('2026-02-11T08:30:00Z'),
balance: NumberDecimal('0.00'),
address: { city: 'SF', zip: '94016' },
tags: [],
ref: 42, // a second type for this field -> inferred "mixed"
// age intentionally absent -> inferred nullable
},
{
email: 'linus@example.com',
age: 27,
active: true,
created: new Date('2026-03-01T12:00:00Z'),
balance: NumberDecimal('9.99'),
address: { city: 'Austin', zip: '73301' },
tags: ['beta'],
ref: null,
},
]);
const userIds = app.users.find({}, { _id: 1 }).toArray().map((u) => u._id);
app.orders.insertMany([
{ user_id: userIds[0], total: 120.5, status: 'paid', placed: new Date('2026-03-02T09:00:00Z') },
{ user_id: userIds[0], total: 9.99, status: 'pending', placed: new Date('2026-03-05T14:00:00Z') },
{ user_id: userIds[1], total: 50.25, status: 'paid', placed: new Date('2026-03-06T16:00:00Z') },
]);
// A view, to confirm introspection does not issue a count command on it
// (MongoDB rejects count on a view with CommandNotSupportedOnView).
app.createView('active_users', 'users', [{ $match: { active: true } }]);
print('users: ' + app.users.countDocuments());
print('orders: ' + app.orders.countDocuments());
print('collections: ' + app.getCollectionNames().join(', '));

View file

@ -1,53 +0,0 @@
// Deterministic, no-LLM smoke for the MongoDB connector. Drives the same
// introspection entry point ktx ingest's "database schema" stage uses, against
// the seeded example database, and asserts the inferred schema.
//
// Usage: node introspect-smoke.mjs [mongoUrl]
import { fileURLToPath } from 'node:url';
import { dirname, resolve } from 'node:path';
const here = dirname(fileURLToPath(import.meta.url));
const ktxRoot = resolve(here, '../../..');
const connectorUrl = `file://${resolve(
ktxRoot,
'packages/cli/dist/connectors/mongodb/live-database-introspection.js',
)}`;
const mongoUrl = process.argv[2] ?? 'mongodb://localhost:27117/app';
const { createMongoDbLiveDatabaseIntrospection } = await import(connectorUrl);
function assert(condition, message) {
if (!condition) {
throw new Error(`assertion failed: ${message}`);
}
}
const port = createMongoDbLiveDatabaseIntrospection({
connections: { 'mongo-example': { driver: 'mongodb', url: mongoUrl, databases: ['app'] } },
});
const snapshot = await port.extractSchema('mongo-example');
const tables = new Map(snapshot.tables.map((table) => [table.name, table]));
assert(snapshot.driver === 'mongodb', 'snapshot driver is mongodb');
assert(['orders', 'users'].every((name) => tables.has(name)), 'users and orders collections introspected');
const users = tables.get('users');
const columns = new Map(users.columns.map((column) => [column.name, column]));
assert(columns.get('_id')?.primaryKey === true && columns.get('_id')?.nullable === false, '_id is the non-null primary key');
assert(columns.get('age')?.nullable === true, 'age is nullable (absent in one document)');
assert(columns.get('email')?.nullable === false, 'email is non-nullable (present in every document)');
assert(columns.get('address')?.normalizedType === 'json', 'nested address maps to opaque json');
assert(columns.get('tags')?.normalizedType === 'json', 'array tags maps to opaque json');
assert(columns.get('ref')?.nativeType === 'mixed', 'ref with two types is inferred as mixed');
const view = tables.get('active_users');
assert(view?.kind === 'view', 'active_users is a view');
assert(view?.estimatedRows === null, 'a view is introspected without a count (estimatedRows null)');
console.log(`OK: introspected ${snapshot.tables.length} collections from ${mongoUrl}`);
for (const table of snapshot.tables) {
console.log(` - ${table.db}.${table.name} (${table.kind}, ${table.columns.length} columns)`);
}
process.exit(0);

View file

@ -1,37 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# Manual smoke for the MongoDB connector: start MongoDB, seed it, and assert the
# connector's schema introspection (the deterministic, no-LLM half of ktx ingest's
# "database schema" stage). The full enrichment ingest is documented in README.md.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
EXAMPLE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
KTX_ROOT="$(cd "$EXAMPLE_DIR/../.." && pwd)"
COMPOSE_FILE="$EXAMPLE_DIR/docker-compose.yml"
CONNECTOR="$KTX_ROOT/packages/cli/dist/connectors/mongodb/live-database-introspection.js"
MONGO_URL="${KTX_MONGODB_URL:-mongodb://localhost:27117/app}"
# Compose engine: docker by default, override for podman:
# KTX_MONGODB_COMPOSE="podman compose" examples/mongodb/scripts/smoke.sh
COMPOSE="${KTX_MONGODB_COMPOSE:-docker compose}"
cleanup() {
if [[ "${KTX_MONGODB_KEEP:-0}" != "1" ]]; then
$COMPOSE -f "$COMPOSE_FILE" down -v >/dev/null 2>&1 || true
fi
}
trap cleanup EXIT
if [[ ! -f "$CONNECTOR" ]]; then
echo "Build the CLI first: pnpm --filter @kaelio/ktx run build" >&2
exit 1
fi
echo "Starting MongoDB and seeding (${COMPOSE})…"
$COMPOSE -f "$COMPOSE_FILE" up -d --wait
echo "Asserting connector introspection against ${MONGO_URL}"
node "$SCRIPT_DIR/introspect-smoke.mjs" "$MONGO_URL"
echo "Smoke passed."

View file

@ -1,11 +1,11 @@
# Orbit-style relationship discovery verification
This **ktx** project backs the default `relationships:verify-orbit` command. It uses
This KTX project backs the default `relationships:verify-orbit` command. It uses
the checked-in Orbit-style SQLite fixture from the relationship discovery
benchmark corpus, with no declared primary keys or foreign keys in the database
schema.
Run from the **ktx** workspace root:
Run from the KTX workspace root:
```bash
pnpm run relationships:verify-orbit

View file

@ -6,6 +6,7 @@ storage:
state: sqlite
search: sqlite-fts5
git:
auto_commit: true
author: "ktx <ktx@example.com>"
ingest:
adapters: []

View file

@ -14,7 +14,7 @@ generated local project.
The managed Python runtime smoke requires `uv` on `PATH`, isolates
`KTX_RUNTIME_ROOT`, verifies `ktx admin runtime status`, runs `ktx sl query --yes` to
install the core runtime from the bundled wheel, checks `ktx admin runtime status`,
starts and reuses the **ktx** daemon, and stops it.
starts and reuses the KTX daemon, and stops it.
The artifact manifest contains the public `@kaelio/ktx` npm tarball and the
bundled `kaelio-ktx` runtime wheel. The smoke does not install standalone

View file

@ -17,19 +17,19 @@ unchanged bounded pattern shards do not schedule LLM work.
## Prerequisites
- Docker with Compose v2
- Node and pnpm matching the **ktx** workspace
- `uv` on `PATH` so the **ktx**-managed Python runtime can install the bundled
- Node and pnpm matching the KTX workspace
- `uv` on `PATH` so the KTX-managed Python runtime can install the bundled
runtime wheel
## Run
From the **ktx** repository root:
From the KTX repository root:
```bash
examples/postgres-historic/scripts/smoke.sh
```
The smoke creates a temporary **ktx** project, isolates the managed Python runtime
The smoke creates a temporary KTX project, isolates the managed Python runtime
under the temporary project parent, starts Postgres on `127.0.0.1:55432`, and
uses this connection URL:
@ -41,7 +41,7 @@ Set `KTX_POSTGRES_HISTORIC_KEEP_DOCKER=1` to leave the container running after
the script exits.
The smoke validates the query-history raw snapshot path without requiring LLM
credentials. It uses **ktx**'s local stage-only ingest API after `ktx setup`, so the
credentials. It uses KTX's local stage-only ingest API after `ktx setup`, so the
deterministic reader, batch SQL parser, stable artifact writer, and diff-based
WorkUnit planning are checked independently from curation.
@ -124,6 +124,6 @@ table.
- Missing grants: confirm `GRANT pg_read_all_stats TO ktx_reader;`.
- Empty snapshot: rerun `scripts/generate-workload.sh base` and keep
`--query-history-min-executions 2` for the smoke.
- SQL-analysis failures: run `pnpm run ktx -- dev runtime status` from the **ktx**
- SQL-analysis failures: run `pnpm run ktx -- dev runtime status` from the KTX
repository root and confirm `uv`, the bundled Python wheel, and the managed
runtime all pass.

View file

@ -17,9 +17,7 @@
"test/**/*.test-utils.ts",
"test/**/acceptance-fixtures.ts",
"src/context/scan/relationship-benchmarks.ts!",
"src/context/scan/relationship-benchmark-report.ts!",
"src/connectors/sqlite/read-query-child.ts!",
"src/context/llm/subprocess-generate-object-child.ts!"
"src/context/scan/relationship-benchmark-report.ts!"
]
},
"docs-site": {
@ -39,10 +37,6 @@
"@semantic-release/release-notes-generator",
"conventional-changelog-conventionalcommits"
],
"ignore": [
".context/**",
"examples/**"
],
"ignoreBinaries": [
"uv",
"lsof"

View file

@ -1,10 +1,10 @@
{
"name": "ktx-workspace",
"version": "0.16.0",
"version": "0.7.0",
"description": "Workspace root for ktx packages",
"private": true,
"type": "module",
"packageManager": "pnpm@11.4.0",
"packageManager": "pnpm@11.1.1",
"engines": {
"node": ">=22.0.0",
"pnpm": ">=10.20.0"
@ -24,7 +24,6 @@
"dead-code:fix": "biome check . --formatter-enabled=false --assist-enabled=false --write && knip --fix --format",
"dead-code:knip": "knip --reporter compact",
"dead-code:knip:production": "knip --production --reporter compact",
"deps:upgrade": "node scripts/upgrade-dependencies.mjs",
"docs": "kill $(lsof -ti:3000) 2>/dev/null; pnpm --filter ktx-docs run dev",
"ktx": "node scripts/run-ktx.mjs",
"link:dev": "node scripts/link-dev-cli.mjs",
@ -32,7 +31,6 @@
"setup:dev": "node scripts/setup-dev.mjs",
"release:published-smoke": "node scripts/published-package-smoke.mjs --require-config",
"release:local-embeddings-smoke": "node scripts/local-embeddings-runtime-smoke.mjs --require-opt-in",
"release:codex-backend-smoke": "node scripts/codex-backend-live-smoke.mjs",
"release:readiness": "node scripts/release-readiness.mjs",
"release:update-version": "node scripts/update-public-release-version.mjs",
"relationships:acquire-public-fixtures": "node scripts/acquire-public-benchmark-fixtures.mjs",
@ -60,15 +58,20 @@
"@semantic-release/github": "^12.0.8",
"@semantic-release/npm": "^13.1.5",
"@semantic-release/release-notes-generator": "^14.1.1",
"@types/node": "^25.9.1",
"@types/node": "^25.7.0",
"better-sqlite3": "^12.10.0",
"conventional-changelog-conventionalcommits": "^9.3.1",
"knip": "^6.14.1",
"pg": "^8.21.0",
"knip": "^6.12.2",
"pg": "^8.20.0",
"semantic-release": "^25.0.3",
"typescript": "^6.0.3",
"yaml": "^2.9.0"
},
"pnpm": {
"onlyBuiltDependencies": [
"better-sqlite3"
]
},
"license": "Apache-2.0",
"repository": {
"type": "git",

View file

@ -1,11 +1,7 @@
{
"name": "@kaelio/ktx",
"version": "0.16.0",
"version": "0.7.0",
"description": "Standalone ktx context layer for data agents",
"author": {
"name": "Kaelio",
"url": "https://www.kaelio.com"
},
"type": "module",
"engines": {
"node": ">=22.0.0"
@ -47,46 +43,36 @@
"search:pglite-sl-prototype": "node ../../scripts/pglite-sl-search-prototype.mjs"
},
"dependencies": {
"@ai-sdk/anthropic": "3.0.78",
"@ai-sdk/devtools": "0.0.18",
"@ai-sdk/google-vertex": "^4.0.134",
"@anthropic-ai/claude-agent-sdk": "0.3.146",
"@aws-sdk/client-athena": "^3.1068.0",
"@aws-sdk/client-glue": "^3.1068.0",
"@clack/core": "1.3.1",
"@ai-sdk/anthropic": "3.0.77",
"@ai-sdk/devtools": "0.0.17",
"@ai-sdk/google-vertex": "^4.0.128",
"@anthropic-ai/claude-agent-sdk": "0.3.142",
"@clack/prompts": "1.4.0",
"@clickhouse/client": "^1.18.5",
"@clickhouse/client": "^1.18.4",
"@commander-js/extra-typings": "14.0.0",
"@duckdb/node-api": "1.5.3-r.3",
"@google-cloud/bigquery": "^8.3.1",
"google-auth-library": "10.6.2",
"@looker/sdk": "^26.8.0",
"@looker/sdk-node": "^26.8.0",
"@looker/sdk-rtl": "^21.6.5",
"@modelcontextprotocol/sdk": "^1.29.0",
"@notionhq/client": "^5.22.0",
"@openai/codex-sdk": "^0.133.0",
"ai": "^6.0.188",
"@notionhq/client": "^5.21.0",
"ai": "^6.0.180",
"better-sqlite3": "^12.10.0",
"commander": "14.0.3",
"fflate": "^0.8.3",
"fflate": "^0.8.2",
"handlebars": "^4.7.9",
"ink": "^7.0.3",
"ink": "^7.0.2",
"lookml-parser": "7.1.0",
"minimatch": "^10.2.5",
"mongodb": "^6.12.0",
"mssql": "^12.5.4",
"mssql": "^12.5.2",
"mysql2": "^3.22.3",
"openai": "^6.38.0",
"openai": "^6.37.0",
"p-limit": "^7.3.0",
"pg": "^8.21.0",
"pino": "^10.3.1",
"pino-pretty": "^13.1.3",
"posthog-node": "^5.34.9",
"pg": "^8.20.0",
"posthog-node": "^5.0.0",
"react": "^19.2.6",
"semver": "^7.8.1",
"simple-git": "3.36.0",
"snowflake-sdk": "^2.4.2",
"snowflake-sdk": "^2.4.1",
"yaml": "^2.9.0",
"zod": "^4.4.3"
},
@ -95,15 +81,14 @@
"@electric-sql/pglite-socket": "^0.1.5",
"@types/better-sqlite3": "^7.6.13",
"@types/mssql": "^12.3.0",
"@types/node": "^25.9.1",
"@types/node": "^25.7.0",
"@types/pg": "^8.20.0",
"@types/react": "^19.2.15",
"@types/semver": "^7.7.1",
"@vitest/coverage-v8": "^4.1.7",
"@types/react": "^19.2.14",
"@vitest/coverage-v8": "^4.1.6",
"ajv": "8.20.0",
"ink-testing-library": "^4.0.0",
"typescript": "^6.0.3",
"vitest": "^4.1.7"
"vitest": "^4.1.6"
},
"license": "Apache-2.0",
"repository": {

View file

@ -7,17 +7,10 @@ const promptsSource = join(packageRoot, 'src', 'prompts');
const promptsTarget = join(packageRoot, 'dist', 'prompts');
const skillsSource = join(packageRoot, 'src', 'skills');
const skillsTarget = join(packageRoot, 'dist', 'skills');
// Per-dialect SQL notes are markdown served by the sql_dialect_notes MCP tool;
// tsc does not emit non-.ts files, so copy them next to their compiled module.
const dialectNotesSource = join(packageRoot, 'src', 'context', 'sql-analysis', 'dialects');
const dialectNotesTarget = join(packageRoot, 'dist', 'context', 'sql-analysis', 'dialects');
await rm(promptsTarget, { recursive: true, force: true });
await rm(skillsTarget, { recursive: true, force: true });
await rm(dialectNotesTarget, { recursive: true, force: true });
await mkdir(dirname(promptsTarget), { recursive: true });
await mkdir(dirname(skillsTarget), { recursive: true });
await mkdir(dirname(dialectNotesTarget), { recursive: true });
await cp(promptsSource, promptsTarget, { recursive: true });
await cp(skillsSource, skillsTarget, { recursive: true });
await cp(dialectNotesSource, dialectNotesTarget, { recursive: true });

View file

@ -24,7 +24,7 @@ export function registerAdminCommands(program: Command, context: KtxCliCommandCo
admin
.command('init')
.description('Initialize a Git-backed ktx project directory for maintenance scripts')
.description('Initialize a Git-backed KTX project directory for maintenance scripts')
.argument('[directory]', 'Project directory')
.option('--force', 'Rewrite ktx.yaml and scaffold files in an existing project', false)
.action(

View file

@ -3,30 +3,6 @@ import type { KtxCliIo } from './cli-runtime.js';
const ESC = String.fromCharCode(0x1b);
export interface CliStyleEnv {
NO_COLOR?: string;
TERM?: string;
}
function ansiEnabled(env: CliStyleEnv = process.env): boolean {
return !env.NO_COLOR && env.TERM !== 'dumb';
}
function ansiColor(text: string, open: number, close: number, env?: CliStyleEnv): string {
if (!ansiEnabled(env)) {
return text;
}
return `${ESC}[${open}m${text}${ESC}[${close}m`;
}
export function dim(text: string, env?: CliStyleEnv): string {
return ansiColor(text, 2, 22, env);
}
export function cyan(text: string, env?: CliStyleEnv): string {
return ansiColor(text, 36, 39, env);
}
export interface RailBufferedSource {
stdoutText(): string;
stderrText(): string;
@ -81,39 +57,27 @@ class KtxCliPromptCancelledError extends Error {
}
export function createClackSpinner(): KtxCliSpinner {
// clack colors the animated spinner frame magenta by default; styleFrame
// (typed in SpinnerOptions, absent from the README) recolors it ktx orange.
return spinner({ styleFrame: orange });
return spinner();
}
// ktx mascot orange (#FF8A4C) via 24-bit truecolor.
function orange(text: string): string {
if (!ansiEnabled()) {
return text;
}
return `${ESC}[38;2;255;138;76m${text}${ESC}[39m`;
function magenta(text: string): string {
return `${ESC}[35m${text}${ESC}[39m`;
}
function red(text: string): string {
return ansiColor(text, 31, 39);
return `${ESC}[31m${text}${ESC}[39m`;
}
/**
* Stderr-only, non-animated spinner. Use this instead of {@link createCliSpinner}
* when the next step reads stdin in raw mode (an Ink TUI or a keypress wait):
* the animated clack spinner seizes stdin via `@clack/core`'s `block()` and
* leaves it dirty, which the following raw-mode reader misreads as a stray key.
*/
export function createStaticCliSpinner(io: KtxCliSpinnerIo): KtxCliSpinner {
return {
start(message) {
io.stderr.write(`${orange('◐')} ${message}\n`);
io.stderr.write(`${magenta('◐')} ${message}\n`);
},
message(message) {
io.stderr.write(`${orange('│')} ${message}\n`);
io.stderr.write(`${magenta('│')} ${message}\n`);
},
stop(message) {
io.stderr.write(`${orange('◇')} ${message}\n`);
io.stderr.write(`${magenta('◇')} ${message}\n`);
},
error(message) {
io.stderr.write(`${red('■')} ${message}\n`);
@ -121,30 +85,6 @@ export function createStaticCliSpinner(io: KtxCliSpinnerIo): KtxCliSpinner {
};
}
/**
* Animated spinner in an interactive terminal, static `◐/◇/■` lines otherwise
* (scripts, CI, piped output) so logs stay clean and uncluttered by frames.
*/
export function createCliSpinner(io: KtxCliIo): KtxCliSpinner {
return io.stdout.isTTY === true ? createClackSpinner() : createStaticCliSpinner(io);
}
export async function runWithCliSpinner<T>(
spinner: KtxCliSpinner,
text: { start: string; success: string; failure: string },
run: () => Promise<T>,
): Promise<T> {
spinner.start(text.start);
try {
const value = await run();
spinner.stop(text.success);
return value;
} catch (error) {
spinner.error(text.failure);
throw error;
}
}
export function createClackPromptAdapter(): KtxCliPromptAdapter {
return {
async confirm(options) {

View file

@ -21,7 +21,7 @@ export function formatClaudeCodePromptCachingWarning(fields: string[]): string |
if (fields.length === 0) {
return null;
}
return `claude-code ignores ${fields.join(', ')} because the Claude Agent SDK does not expose ktx prompt-cache TTL, tool, or history markers.`;
return `claude-code ignores ${fields.join(', ')} because the Claude Agent SDK does not expose KTX prompt-cache TTL, tool, or history markers.`;
}
export function formatClaudeCodePromptCachingFix(): string {

View file

@ -2,8 +2,6 @@ import { existsSync } from 'node:fs';
import { join } from 'node:path';
import { Command, type CommandUnknownOpts, InvalidArgumentError } from '@commander-js/extra-typings';
import type { KtxCliDeps, KtxCliIo, KtxCliPackageInfo } from './cli-runtime.js';
import { SLACK_HELP_FOOTER, writeErrorCommunityHint } from './community-cta.js';
import { registerCompletionCommands } from './commands/completion-commands.js';
import { registerConnectionCommands } from './commands/connection-commands.js';
import { registerIngestCommands } from './commands/ingest-commands.js';
import { registerWikiCommands } from './commands/knowledge-commands.js';
@ -17,7 +15,6 @@ import { renderMissingProjectMessage } from './doctor.js';
import { findNearestKtxProjectDir, resolveKtxProjectDir } from './project-resolver.js';
import { profileMark, profileSpan } from './startup-profile.js';
import type { CommandOutcome } from './telemetry/index.js';
import { prepareUpdateCheckNotice, type PrepareUpdateCheckNoticeOptions } from './update-check/update-check.js';
profileMark('module:cli-program');
@ -41,8 +38,6 @@ interface KtxCommanderProgramOptions {
runInit: (args: { projectDir: string; force: boolean }, io: KtxCliIo) => Promise<number>;
}
type KtxCliUpdateCheckOptions = Pick<PrepareUpdateCheckNoticeOptions, 'env' | 'fetchDistTags' | 'homeDir' | 'now'>;
export interface BuildKtxProgramOptions {
io: KtxCliIo;
deps: KtxCliDeps;
@ -51,7 +46,6 @@ export interface BuildKtxProgramOptions {
setExitCode?: (code: number) => void;
argv?: string[];
setTelemetryModule?: (telemetry: typeof import('./telemetry/index.js')) => void;
updateCheck?: KtxCliUpdateCheckOptions;
}
type CommanderExitLike = { exitCode: number; code: string; message: string };
@ -133,7 +127,7 @@ export function parseBooleanStringOption(value: string): boolean {
}
export function parseSafeConnectionIdOption(value: string): string {
if (!/^[a-zA-Z0-9_][a-zA-Z0-9_-]*$/.test(value)) {
if (!/^[a-zA-Z0-9][a-zA-Z0-9_-]*$/.test(value)) {
throw new InvalidArgumentError(`Unsafe connection id: ${value}`);
}
return value;
@ -252,14 +246,13 @@ export function resolveCommandProjectDirOverride(command: CommandWithGlobalOptio
function createBaseProgram(info: KtxCliPackageInfo, io: KtxCliIo): Command {
return new Command()
.name('ktx')
.description('ktx data agent context layer CLI')
.option('--project-dir <path>', 'ktx project directory (default: KTX_PROJECT_DIR, nearest ktx.yaml, or cwd)')
.description('KTX data agent context layer CLI')
.option('--project-dir <path>', 'KTX project directory (default: KTX_PROJECT_DIR, nearest ktx.yaml, or cwd)')
.option('--debug', 'Enable diagnostic logging to stderr')
.version(`${info.name} ${info.version}`, '-v, --version', 'Show CLI version')
.helpOption('-h, --help', 'Show this help text')
.configureHelp({ showGlobalOptions: true })
.showHelpAfterError()
.addHelpText('after', `\n${SLACK_HELP_FOOTER}`)
.exitOverride()
.configureOutput({
writeOut: (chunk) => io.stdout.write(chunk),
@ -437,36 +430,18 @@ export function collectCommandFlagsPresent(command: CommandUnknownOpts): Record<
export function buildKtxProgram(options: BuildKtxProgramOptions): Command {
const program = createBaseProgram(options.packageInfo, options.io);
let pendingUpdateNotice: string | null = null;
program.hook('preAction', async (_thisCommand, actionCommand) => {
// The hidden completion command must stay silent and side-effect free: skip
// the telemetry notice, command span, project checks, and update checks entirely.
if (commandPath(actionCommand as CommandPathNode).includes('__complete')) {
return;
}
const commandNode = actionCommand as CommandPathNode;
const updateCheck = await prepareUpdateCheckNotice({
io: options.io,
env: options.updateCheck?.env,
fetchDistTags: options.updateCheck?.fetchDistTags,
homeDir: options.updateCheck?.homeDir,
installedVersion: options.packageInfo.version,
now: options.updateCheck?.now,
commandOptions: commandOptions(commandNode),
});
pendingUpdateNotice = updateCheck.notice;
const telemetry = await import('./telemetry/index.js');
options.setTelemetryModule?.(telemetry);
await telemetry.showTelemetryNoticeIfNeeded(options.io, options.packageInfo);
const commandNode = actionCommand as CommandPathNode;
const path = commandPath(commandNode);
const projectDir = resolveCommandProjectDir(commandNode);
const hasProject = ktxYamlExists(projectDir);
const attachProjectGroup = shouldAttachCommandProjectGroup(path, hasProject);
telemetry.beginCommandSpan({
commandPath: path,
flagsPresent: collectCommandFlagsPresent(actionCommand),
flagsPresent: collectCommandFlagsPresent(commandNode as unknown as CommandUnknownOpts),
projectDir: attachProjectGroup ? projectDir : undefined,
hasProject,
attachProjectGroup,
@ -476,13 +451,6 @@ export function buildKtxProgram(options: BuildKtxProgramOptions): Command {
ensureProjectAvailable(options.io, commandNode);
});
program.hook('postAction', () => {
if (pendingUpdateNotice) {
options.io.stderr.write(pendingUpdateNotice);
pendingUpdateNotice = null;
}
});
const context: KtxCliCommandContext = {
io: options.io,
deps: options.deps,
@ -508,7 +476,6 @@ export function buildKtxProgram(options: BuildKtxProgramOptions): Command {
registerStatusCommands(program, context);
registerMcpCommands(program, context);
registerAdminCommands(program, context);
registerCompletionCommands(program, context);
return program;
}
@ -555,15 +522,7 @@ export async function runCommanderKtxCli(
try {
return await runBareInteractiveCommand(program, io, context);
} catch (error) {
const telemetry = await import('./telemetry/index.js');
await telemetry.reportException({
error,
context: { source: 'bare-interactive', handled: true, fatal: false },
packageInfo: info,
io,
});
io.stderr.write(`${formatCliError(error)}\n`);
writeErrorCommunityHint(io, 'error');
return 1;
}
}
@ -588,7 +547,6 @@ export async function runCommanderKtxCli(
exitCode = error.exitCode === 0 ? 0 : 1;
} else {
io.stderr.write(`${formatCliError(error)}\n`);
writeErrorCommunityHint(io, 'error');
exitCode = 1;
}
} finally {
@ -598,23 +556,6 @@ export async function runCommanderKtxCli(
outcome: commandOutcomeForParseResult(parseError, exitCode),
error: parseError,
});
if (
parseError &&
!isCommanderExit(parseError) &&
!isKtxProjectMissingAbortError(parseError)
) {
await telemetryModule.reportException({
error: parseError,
context: {
source: completed?.commandPath.join(' ') ?? 'commander parseAsync',
handled: true,
fatal: false,
},
projectDir: completed?.projectGroupAttached ? completed.projectDir : undefined,
packageInfo: info,
io,
});
}
await telemetryModule.emitCompletedCommand({ completed, packageInfo: info, io });
await telemetryModule.shutdownTelemetryEmitter();
}

View file

@ -12,7 +12,6 @@ import type { KtxSqlArgs } from './sql.js';
import { profileMark, profileSpan } from './startup-profile.js';
import type { KtxTextIngestArgs } from './text-ingest.js';
import { assertCliVersion } from './release-version.js';
import { writeErrorCommunityHint } from './community-cta.js';
profileMark('module:cli-runtime');
@ -61,7 +60,7 @@ export function packageInfoFromJson(packageJson: unknown): KtxCliPackageInfo {
typeof packageJson.name !== 'string' ||
typeof packageJson.version !== 'string'
) {
throw new Error('Invalid ktx CLI package metadata');
throw new Error('Invalid KTX CLI package metadata');
}
return {
@ -77,7 +76,7 @@ async function runInit(args: { projectDir: string; force: boolean }, io: KtxCliI
force: args.force,
});
io.stdout.write(`Initialized ktx project at ${result.projectDir}\n`);
io.stdout.write(`Initialized KTX project at ${result.projectDir}\n`);
io.stdout.write(`Config: ${result.configPath}\n`);
io.stdout.write(`Commit: ${result.commitHash ?? 'none'}\n`);
return 0;
@ -90,94 +89,6 @@ export async function runInitForCommander(
return await runInit(args, io);
}
function signalExitCode(signal: NodeJS.Signals): number {
// 128 + signal number: SIGINT (2) -> 130, SIGTERM (15) -> 143.
return signal === 'SIGTERM' ? 143 : 130;
}
/**
* Flush telemetry on interrupt for the real CLI process. `capture()` is
* fire-and-forget and the only flush guarantee lives in a `finally` a signal
* skips, so Ctrl-C / `kill` of a long-running command (ingest, `mcp stdio`)
* would otherwise drop its `command` event and queued events. Installed only
* when driving the actual process; programmatic/test callers pass their own
* `io` and never reach here. Returns a disposer that removes the listeners.
*/
function installTelemetrySignalFlush(io: KtxCliIo, info: KtxCliPackageInfo): () => void {
let handling = false;
const handle = (signal: NodeJS.Signals): void => {
if (handling) {
process.exit(signalExitCode(signal));
}
handling = true;
void (async () => {
try {
const { emitAbortedCommandAndShutdown } = await import('./telemetry/index.js');
await emitAbortedCommandAndShutdown({ packageInfo: info, io });
} catch {
// Best-effort: never let a telemetry hiccup block the interrupt exit.
}
process.exit(signalExitCode(signal));
})();
};
const onSigint = (): void => handle('SIGINT');
const onSigterm = (): void => handle('SIGTERM');
process.on('SIGINT', onSigint);
process.on('SIGTERM', onSigterm);
return () => {
process.off('SIGINT', onSigint);
process.off('SIGTERM', onSigterm);
};
}
/** @internal */
export function createGlobalExceptionReporter(io: KtxCliIo, info: KtxCliPackageInfo) {
return async (source: 'uncaughtException' | 'unhandledRejection', error: unknown): Promise<void> => {
const { reportException, shutdownTelemetryEmitter } = await import('./telemetry/index.js');
await reportException({
error,
context: { source, handled: false, fatal: true },
io,
packageInfo: info,
immediate: true,
});
await shutdownTelemetryEmitter();
};
}
/** @internal */
export function writeGlobalExceptionToStderr(io: KtxCliIo, error: unknown): void {
if (error instanceof Error && error.stack) {
io.stderr.write(`${error.stack}\n`);
} else {
io.stderr.write(`${String(error)}\n`);
}
writeErrorCommunityHint(io, 'crash');
}
export function installGlobalExceptionHandlers(io: KtxCliIo, info: KtxCliPackageInfo): () => void {
const report = createGlobalExceptionReporter(io, info);
const handle = (source: 'uncaughtException' | 'unhandledRejection', error: unknown): void => {
void (async () => {
try {
await report(source, error);
} catch {
// Best-effort: preserve Node's process termination behavior.
}
writeGlobalExceptionToStderr(io, error);
process.exit(1);
})();
};
const onUncaught = (error: Error): void => handle('uncaughtException', error);
const onUnhandled = (reason: unknown): void => handle('unhandledRejection', reason);
process.on('uncaughtException', onUncaught);
process.on('unhandledRejection', onUnhandled);
return () => {
process.off('uncaughtException', onUncaught);
process.off('unhandledRejection', onUnhandled);
};
}
export async function runKtxCli(
argv = process.argv.slice(2),
io: KtxCliIo = process,
@ -187,17 +98,7 @@ export async function runKtxCli(
profileMark('runtime:runKtxCli');
const { runCommanderKtxCli } = await profileSpan('import ./cli-program.js', () => import('./cli-program.js'));
// Real-process entry only: flush telemetry if interrupted. Test/programmatic
// callers pass their own `io`, so they never install process-level handlers.
const removeSignalFlush = (io as unknown) === process ? installTelemetrySignalFlush(io, info) : undefined;
const removeGlobalExceptionHandlers =
(io as unknown) === process ? installGlobalExceptionHandlers(io, info) : undefined;
try {
return await runCommanderKtxCli(argv, io, deps, info, {
runInit: runInitForCommander,
});
} finally {
removeGlobalExceptionHandlers?.();
removeSignalFlush?.();
}
return await runCommanderKtxCli(argv, io, deps, info, {
runInit: runInitForCommander,
});
}

View file

@ -16,11 +16,7 @@ export function walkCommandTree(command: CommandUnknownOpts): CommandTreeNode {
description: command.description(),
aliases: command.aliases(),
arguments: command.registeredArguments.map(formatArgumentDeclaration),
// Internal commands (e.g. the shell-completion helper `__complete`) use a
// `__` prefix and are omitted from the human-facing command tree.
children: command.commands
.filter((child) => !child.name().startsWith('__'))
.map((child) => walkCommandTree(child)),
children: command.commands.map((child) => walkCommandTree(child)),
};
}

View file

@ -1,44 +0,0 @@
import { Argument, type Command } from '@commander-js/extra-typings';
import type { KtxCliCommandContext } from '../cli-program.js';
import { computeCompletions } from '../completion/complete-engine.js';
import { completionScript } from '../completion/completion-scripts.js';
import { createProjectCompletionProviders } from '../completion/dynamic-candidates.js';
import { profileMark } from '../startup-profile.js';
profileMark('module:commands/completion-commands');
export function registerCompletionCommands(program: Command, context: KtxCliCommandContext): void {
program
.command('completion')
.description('Print a shell completion script for ktx')
.addArgument(new Argument('<shell>', 'Target shell').choices(['zsh', 'bash']))
.addHelpText(
'after',
'\nEnable completion by adding the matching line to your shell startup file:\n' +
' zsh: eval "$(ktx completion zsh)"\n' +
' bash: eval "$(ktx completion bash)"\n',
)
.action((shell) => {
context.io.stdout.write(completionScript(shell));
});
// Hidden command invoked by the generated shell scripts. It must only ever
// print newline-separated candidates to stdout and exit 0, so a TAB press is
// never disrupted by an error, a telemetry notice, or a parse failure.
program
.command('__complete', { hidden: true })
.argument('[words...]')
.allowUnknownOption(true)
.helpOption(false)
.action(async (words: string[]) => {
try {
const candidates = await computeCompletions(program, words, createProjectCompletionProviders());
if (candidates.length > 0) {
context.io.stdout.write(`${candidates.join('\n')}\n`);
}
} catch {
// Swallow: completion must never break the shell.
}
context.setExitCode(0);
});
}

View file

@ -37,7 +37,7 @@ export function registerConnectionCommands(program: Command, context: KtxCliComm
connection
.command('test')
.description('Test one or all configured connections (default: all)')
.argument('[connectionId]', 'ktx connection id to test (omit to test all)')
.argument('[connectionId]', 'KTX connection id to test (omit to test all)')
.option('--all', 'Test every configured connection and print a summary list')
.action(async (connectionId: string | undefined, options: { all?: boolean }, command) => {
if (options.all === true && connectionId !== undefined) {

View file

@ -1,12 +1,10 @@
import { type Command, InvalidArgumentError, Option } from '@commander-js/extra-typings';
import { type Command, Option } from '@commander-js/extra-typings';
import {
collectOption,
type KtxCliCommandContext,
parsePositiveIntegerOption,
resolveCommandProjectDir,
} from '../cli-program.js';
import { KTX_SCAN_ENRICHMENT_STAGES } from '../context/scan/enrichment-state.js';
import type { KtxScanEnrichmentStage } from '../context/scan/types.js';
import type { KtxCliDeps, KtxCliIo } from '../index.js';
import { runtimeInstallPolicyFromFlags } from '../managed-python-command.js';
import type { KtxPublicIngestArgs } from '../public-ingest.js';
@ -16,36 +14,6 @@ import { resolveConnectionSelection } from './connection-selection.js';
profileMark('module:commands/ingest-commands');
/**
* Parses `--stages` into an ordered, de-duplicated subset of the canonical
* enrichment-stage registry. An unknown or empty name is a hard parse error so
* a typo never silently degrades to "run everything."
*
* @internal
*/
export function parseEnrichmentStagesOption(value: string): KtxScanEnrichmentStage[] {
const names = value
.split(',')
.map((name) => name.trim())
.filter((name) => name.length > 0);
if (names.length === 0) {
throw new InvalidArgumentError(
`must be a non-empty comma-separated list of stages (${KTX_SCAN_ENRICHMENT_STAGES.join(', ')})`,
);
}
const valid = new Set<string>(KTX_SCAN_ENRICHMENT_STAGES);
const selected = new Set<KtxScanEnrichmentStage>();
for (const name of names) {
if (!valid.has(name)) {
throw new InvalidArgumentError(
`unknown stage "${name}"; valid stages are ${KTX_SCAN_ENRICHMENT_STAGES.join(', ')}`,
);
}
selected.add(name as KtxScanEnrichmentStage);
}
return KTX_SCAN_ENRICHMENT_STAGES.filter((stage) => selected.has(stage));
}
interface IngestCommandOptions {
runTextIngest: (args: KtxTextIngestArgs, io: KtxCliIo, deps: KtxCliDeps) => Promise<number>;
}
@ -57,26 +25,18 @@ export function registerIngestCommands(
): void {
const ingest = program
.command('ingest')
.description('Build or inspect ktx context, or capture text into memory')
.description('Build or inspect KTX context, or capture text into memory')
.usage('[options] [connectionId]')
.argument('[connectionId]', 'Configured connection id to ingest (omit to ingest all)')
.option('--all', 'Ingest all configured connections', false)
.addOption(new Option('--fast', 'Use deterministic database schema ingest').conflicts('deep'))
.addOption(new Option('--deep', 'Use AI-enriched database ingest').conflicts('fast'))
.addOption(new Option('--query-history', 'Include database query-history usage patterns').conflicts('noQueryHistory'))
.addOption(new Option('--no-query-history', 'Skip database query-history usage patterns'))
.option('--query-history-window-days <days>', 'Query-history lookback window for this run', parsePositiveIntegerOption)
.option(
'--stages <stages>',
'Comma-separated enrichment stages to (re)run (descriptions,embeddings,relationships); omit to run all',
parseEnrichmentStagesOption,
)
.option('--text <content>', 'Capture inline text into ktx memory; repeatable', collectOption, [])
.option('--file <path>', 'Capture a text file into ktx memory; use - for stdin; repeatable', collectOption, [])
.option(
'--verbatim',
'Store each --text/--file document body unchanged as a GLOBAL wiki page; the LLM derives only metadata',
false,
)
.option('--connection-id <connectionId>', 'ktx connection id to tag captured text/file notes')
.option('--text <content>', 'Capture inline text into KTX memory; repeatable', collectOption, [])
.option('--file <path>', 'Capture a text file into KTX memory; use - for stdin; repeatable', collectOption, [])
.option('--connection-id <connectionId>', 'KTX connection id to tag captured text/file notes')
.option('--user-id <id>', 'Memory user id for text/file capture attribution', 'local-cli')
.option('--fail-fast', 'Stop after the first failed text/file item', false)
.addOption(new Option('--plain', 'Print plain text output').conflicts(['json']))
@ -89,14 +49,6 @@ export function registerIngestCommands(
const projectDir = resolveCommandProjectDir(command);
const hasTextCapture = options.text.length > 0 || options.file.length > 0;
if (options.verbatim === true && !hasTextCapture) {
command.error('error: --verbatim requires --text or --file');
}
if (options.stages !== undefined && hasTextCapture) {
command.error('error: --stages applies to database ingest only; it cannot be combined with --text or --file');
}
if (hasTextCapture) {
if (connectionId !== undefined) {
command.error(
@ -116,7 +68,6 @@ export function registerIngestCommands(
userId: options.userId,
json: options.json === true,
failFast: options.failFast === true,
...(options.verbatim === true ? { verbatim: true } : {}),
},
context.io,
context.deps,
@ -136,9 +87,10 @@ export function registerIngestCommands(
all: selection.kind === 'all',
json: options.json === true,
inputMode: options.input === false ? 'disabled' : 'auto',
...(options.fast === true ? { depth: 'fast' as const } : {}),
...(options.deep === true ? { depth: 'deep' as const } : {}),
queryHistory,
...(options.queryHistoryWindowDays !== undefined ? { queryHistoryWindowDays: options.queryHistoryWindowDays } : {}),
...(options.stages ? { stages: options.stages } : {}),
cliVersion: context.packageInfo.version,
runtimeInstallPolicy: runtimeInstallPolicyFromFlags(options),
};

View file

@ -21,13 +21,12 @@ function isDebugEnabled(command: CommandWithGlobalOptions): boolean {
}
export function registerWikiCommands(program: Command, context: KtxCliCommandContext): void {
const wiki = program
program
.command('wiki')
.description('List, search, or read local wiki pages')
.description('List or search local wiki pages')
.usage('[options] [query...]')
.argument('[query...]', 'Search query; omit to list all pages')
.option('--user-id <id>', 'Local user id', 'local')
.option('-c, --connection <id>', 'Scope results to one connection (unscoped pages plus pages tagged with it)')
.option('--limit <number>', 'Maximum search results (search mode only)', parsePositiveIntegerOption)
.addOption(
new Option('--output <mode>', 'Output mode: pretty (default in TTY), plain (TSV), or json').choices([
@ -47,7 +46,6 @@ export function registerWikiCommands(program: Command, context: KtxCliCommandCon
query: string[],
options: {
userId: string;
connection?: string;
limit?: number;
output?: 'pretty' | 'plain' | 'json';
json?: boolean;
@ -59,7 +57,6 @@ export function registerWikiCommands(program: Command, context: KtxCliCommandCon
command: 'list',
projectDir: resolveCommandProjectDir(command),
userId: options.userId,
...(options.connection !== undefined ? { connectionId: options.connection } : {}),
output: options.output,
json: options.json,
cliVersion: context.packageInfo.version,
@ -71,7 +68,6 @@ export function registerWikiCommands(program: Command, context: KtxCliCommandCon
projectDir: resolveCommandProjectDir(command),
query: query.join(' '),
userId: options.userId,
...(options.connection !== undefined ? { connectionId: options.connection } : {}),
output: options.output,
json: options.json,
...(isDebugEnabled(command) ? { debug: true } : {}),
@ -80,18 +76,4 @@ export function registerWikiCommands(program: Command, context: KtxCliCommandCon
});
},
);
wiki
.command('read')
.description('Read a wiki page file by key')
.argument('<key>', 'Wiki page key')
.action(async (key: string, _options, command) => {
const parentOpts = command.parent?.opts() as { userId?: string } | undefined;
await runKnowledgeArgs(context, {
command: 'read',
projectDir: resolveCommandProjectDir(command),
key,
userId: parentOpts?.userId ?? 'local',
});
});
}

View file

@ -27,11 +27,11 @@ function binPath(): string {
function formatMcpStartResultMessage(input: { status: 'started' | 'already-running'; url: string }): string {
return [
input.status === 'started' ? `ktx MCP daemon started: ${input.url}` : `ktx MCP daemon already running: ${input.url}`,
input.status === 'started' ? `KTX MCP daemon started: ${input.url}` : `KTX MCP daemon already running: ${input.url}`,
'',
'ktx is ready for configured agents.',
'Open your agent for this ktx project and ask a data question, for example:',
' "Use ktx to show me the available tables and metrics."',
'KTX is ready for configured agents.',
'Open your agent for this KTX project and ask a data question, for example:',
' "Use KTX to show me the available tables and metrics."',
'',
].join('\n');
}
@ -50,14 +50,14 @@ async function printMcpStatus(context: KtxCliCommandContext, projectDir: string)
export function registerMcpCommands(program: Command, context: KtxCliCommandContext): void {
const mcp = program
.command('mcp')
.description('Manage the ktx MCP HTTP server (bare command: show status)')
.description('Manage the KTX MCP HTTP server (bare command: show status)')
.action(async (_options, command) => {
await printMcpStatus(context, resolveCommandProjectDir(command));
});
mcp
.command('stdio')
.description('Run the ktx MCP server over stdio')
.description('Run the KTX MCP server over stdio')
.action(async (_options, command) => {
await (context.deps.mcp?.runStdioServer ?? runKtxMcpStdioServer)({
projectDir: resolveCommandProjectDir(command),
@ -68,7 +68,7 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
mcp
.command('start')
.description('Start the ktx MCP HTTP server')
.description('Start the KTX MCP HTTP server')
.option('--host <host>', 'Host to bind', '127.0.0.1')
.option('--port <n>', 'Port to bind', parsePositiveIntegerOption, 7878)
.option('--token <token>', 'Bearer token required for non-loopback binding')
@ -96,7 +96,7 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
allowedOrigins: options.allowedOrigin,
io: context.io,
});
context.io.stdout.write(`ktx MCP server listening at http://${options.host}:${options.port}/mcp\n`);
context.io.stdout.write(`KTX MCP server listening at http://${options.host}:${options.port}/mcp\n`);
return;
}
const result = await (context.deps.mcp?.startDaemon ?? startKtxMcpDaemon)({
@ -114,24 +114,24 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
mcp
.command('stop')
.description('Stop the ktx MCP daemon')
.description('Stop the KTX MCP daemon')
.action(async (_options, command) => {
const result = await (context.deps.mcp?.stopDaemon ?? stopKtxMcpDaemon)({
projectDir: resolveCommandProjectDir(command),
});
context.io.stdout.write(result.status === 'stopped' ? 'ktx MCP daemon stopped.\n' : 'ktx MCP daemon is not running.\n');
context.io.stdout.write(result.status === 'stopped' ? 'KTX MCP daemon stopped.\n' : 'KTX MCP daemon is not running.\n');
});
mcp
.command('status')
.description('Show ktx MCP daemon status')
.description('Show KTX MCP daemon status')
.action(async (_options, command) => {
await printMcpStatus(context, resolveCommandProjectDir(command));
});
mcp
.command('logs')
.description('Print the ktx MCP daemon log')
.description('Print the KTX MCP daemon log')
.option('--follow', 'Follow log output', false)
.action(async (options, command) => {
const logPath = mcpDaemonLayout(resolveCommandProjectDir(command)).logPath;

View file

@ -18,7 +18,7 @@ async function runRuntimeArgs(context: KtxCliCommandContext, args: KtxRuntimeArg
export function registerRuntimeCommands(program: Command, context: KtxCliCommandContext): void {
const runtime = program
.command('runtime')
.description('Install, start, stop, and inspect the ktx-managed Python runtime')
.description('Install, start, stop, and inspect the KTX-managed Python runtime')
.showHelpAfterError();
runtime
@ -38,7 +38,7 @@ export function registerRuntimeCommands(program: Command, context: KtxCliCommand
runtime
.command('start')
.description('Start the ktx daemon')
.description('Start the KTX daemon')
.addOption(createRuntimeFeatureOption())
.option('--force', 'Restart even when a matching daemon is already running', false)
.action(async (options: { feature: RuntimeFeature; force?: boolean }, command: CommandWithGlobalOptions) => {
@ -53,8 +53,8 @@ export function registerRuntimeCommands(program: Command, context: KtxCliCommand
runtime
.command('stop')
.description('Stop the ktx daemon')
.option('--all', 'Stop all ktx daemon processes recorded or discoverable on this machine', false)
.description('Stop the KTX daemon')
.option('--all', 'Stop all KTX daemon processes recorded or discoverable on this machine', false)
.action(async (options: { all?: boolean }, command: CommandWithGlobalOptions) => {
await runRuntimeArgs(context, {
command: 'stop',

View file

@ -2,7 +2,7 @@ import { type Command, InvalidArgumentError, Option } from '@commander-js/extra-
import type { KtxCliCommandContext } from '../cli-program.js';
import { resolveCommandProjectDir } from '../cli-program.js';
import type { KtxSetupDatabaseDriver } from '../setup-databases.js';
import { isKtxSetupLlmBackend, type KtxSetupLlmBackend } from '../setup-models.js';
import type { KtxSetupLlmBackend } from '../setup-models.js';
import type { KtxSetupSourceType } from '../setup-sources.js';
async function runSetupArgs(
@ -16,7 +16,7 @@ async function runSetupArgs(
function positiveInteger(value: string): number {
const parsed = Number.parseInt(value, 10);
if (!Number.isInteger(parsed) || parsed <= 0) {
throw new InvalidArgumentError(`Expected a positive integer, received ${value}`);
throw new Error(`Expected a positive integer, received ${value}`);
}
return parsed;
}
@ -29,7 +29,7 @@ function embeddingBackend(value: string): 'openai' | 'sentence-transformers' {
}
function llmBackend(value: string): KtxSetupLlmBackend {
if (isKtxSetupLlmBackend(value)) {
if (value === 'anthropic' || value === 'vertex' || value === 'claude-code') {
return value;
}
throw new InvalidArgumentError(`invalid choice '${value}'`);
@ -38,7 +38,6 @@ function llmBackend(value: string): KtxSetupLlmBackend {
function databaseDriver(value: string): KtxSetupDatabaseDriver {
if (
value === 'sqlite' ||
value === 'duckdb' ||
value === 'postgres' ||
value === 'mysql' ||
value === 'clickhouse' ||
@ -58,9 +57,7 @@ function sourceType(value: string): KtxSetupSourceType {
value === 'metabase' ||
value === 'looker' ||
value === 'lookml' ||
value === 'notion' ||
value === 'sigma' ||
value === 'gdrive'
value === 'notion'
) {
return value;
}
@ -92,13 +89,13 @@ function shouldShowSetupEntryMenu(
target?: string;
global?: boolean;
local?: boolean;
installDir?: string;
skipAgents?: boolean;
yes?: boolean;
input?: boolean;
llmBackend?: KtxSetupLlmBackend;
anthropicApiKeyEnv?: string;
anthropicApiKeyFile?: string;
llmModel?: string;
vertexProject?: string;
vertexLocation?: string;
skipLlm?: boolean;
@ -135,9 +132,6 @@ function shouldShowSetupEntryMenu(
metabaseDatabaseId?: number;
notionCrawlMode?: string;
notionRootPageId?: string[];
gdriveServiceAccountKeyRef?: string;
gdriveFolderId?: string;
gdriveRecursive?: boolean;
skipSources?: boolean;
},
command: Command,
@ -166,13 +160,13 @@ function shouldShowSetupEntryMenu(
'target',
'global',
'local',
'installDir',
'skipAgents',
'yes',
'input',
'llmBackend',
'anthropicApiKeyEnv',
'anthropicApiKeyFile',
'llmModel',
'vertexProject',
'vertexLocation',
'skipLlm',
@ -203,9 +197,6 @@ function shouldShowSetupEntryMenu(
'sourceTarget',
'metabaseDatabaseId',
'notionCrawlMode',
'gdriveServiceAccountKeyRef',
'gdriveFolderId',
'gdriveRecursive',
'skipSources',
].some((optionName) => optionWasSpecified(command, optionName));
}
@ -213,8 +204,8 @@ function shouldShowSetupEntryMenu(
export function registerSetupCommands(program: Command, context: KtxCliCommandContext): void {
const setup = program
.command('setup')
.description('Set up or resume a local ktx project')
.addOption(new Option('--project-dir <path>', 'ktx project directory').hideHelp())
.description('Set up or resume a local KTX project')
.addOption(new Option('--project-dir <path>', 'KTX project directory').hideHelp())
.option('--agents', 'Install agent integration only', false)
.addOption(
new Option('--target <target>', 'Agent target').choices([
@ -228,10 +219,6 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
)
.option('--global', 'Install agent integration into the global target scope', false)
.option('--local', 'Install Claude Code MCP config into the private per-project ~/.claude.json scope', false)
.option(
'--install-dir <path>',
'Directory to install project-scoped agent config into (defaults to the ktx project directory)',
)
.addOption(new Option('--skip-agents', 'Leave agent integration incomplete for now').hideHelp().default(false))
.option('--yes', 'Accept project creation and runtime install defaults where setup confirms', false)
.option('--no-input', 'Disable interactive terminal input')
@ -242,6 +229,7 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
.addOption(
new Option('--anthropic-api-key-file <path>', 'File containing the Anthropic API key').hideHelp(),
)
.addOption(new Option('--llm-model <model>', 'LLM model ID or backend model alias').hideHelp())
.addOption(new Option('--vertex-project <project>', 'Google Vertex AI project ID, env:NAME, or file:/path').hideHelp())
.addOption(new Option('--vertex-location <location>', 'Google Vertex AI location, env:NAME, or file:/path').hideHelp())
.addOption(new Option('--skip-llm', 'Leave LLM setup incomplete for now').hideHelp().default(false))
@ -310,7 +298,7 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
.hideHelp(),
)
.addOption(
new Option('--skip-databases', 'Leave database setup incomplete; ktx cannot work until a database is added')
new Option('--skip-databases', 'Leave database setup incomplete; KTX cannot work until a database is added')
.hideHelp()
.default(false),
)
@ -320,14 +308,9 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
.addOption(new Option('--source-git-url <url>', 'Git URL for dbt, MetricFlow, or LookML').hideHelp())
.addOption(new Option('--source-branch <branch>', 'Git branch for source setup').hideHelp())
.addOption(new Option('--source-subpath <path>', 'Repo subpath for source setup').hideHelp())
.addOption(
new Option(
'--source-auth-token-ref <ref>',
'env: or file: credential ref for source repo auth or Notion integration token',
).hideHelp(),
)
.addOption(new Option('--source-auth-token-ref <ref>', 'env: or file: credential ref for source repo auth').hideHelp())
.addOption(new Option('--source-url <url>', 'Source service URL for Metabase or Looker').hideHelp())
.addOption(new Option('--source-api-key-ref <ref>', 'env: or file: API key ref for Metabase').hideHelp())
.addOption(new Option('--source-api-key-ref <ref>', 'env: or file: API key ref for Metabase or Notion').hideHelp())
.addOption(new Option('--source-client-id <id>', 'Looker client id').hideHelp())
.addOption(new Option('--source-client-secret-ref <ref>', 'env: or file: Looker client secret ref').hideHelp())
.addOption(new Option('--source-warehouse-connection-id <id>', 'Mapped warehouse connection id').hideHelp())
@ -346,12 +329,6 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
.default([] as string[])
.hideHelp(),
)
.addOption(
new Option('--gdrive-service-account-key-ref <ref>', 'file: reference to a Google service account JSON key')
.hideHelp(),
)
.addOption(new Option('--gdrive-folder-id <id>', 'Google Drive folder id to ingest').hideHelp())
.addOption(new Option('--gdrive-recursive', 'Recursively traverse Google Drive subfolders').hideHelp().default(false))
.addOption(new Option('--skip-sources', 'Mark optional source setup complete with no sources').hideHelp().default(false))
.showHelpAfterError();
@ -415,16 +392,6 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
context.setExitCode(1);
return;
}
if (options.installDir && (options.global || options.local)) {
context.io.stderr.write('Choose either --install-dir or a scope flag (--global / --local), not both.\n');
context.setExitCode(1);
return;
}
if (options.installDir && options.target === 'claude-desktop') {
context.io.stderr.write('--install-dir does not apply to --target claude-desktop, which is always global.\n');
context.setExitCode(1);
return;
}
const creatingDatabaseConnection = options.database.length > 0 || options.databaseUrl !== undefined;
if (creatingDatabaseConnection && options.databaseConnectionId.length > 1) {
@ -434,8 +401,6 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
}
const resolvedAgentScope = options.local ? 'local' : options.global ? 'global' : 'project';
const debugEnabled =
((command.optsWithGlobals ? command.optsWithGlobals() : command.opts()) as { debug?: unknown }).debug === true;
await runSetupArgs(context, {
command: 'run',
projectDir: resolveCommandProjectDir(command),
@ -443,15 +408,14 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
agents: options.agents === true,
...(options.target ? { target: options.target } : {}),
agentScope: resolvedAgentScope,
...(options.installDir ? { installRoot: options.installDir } : {}),
skipAgents: options.skipAgents === true,
inputMode: options.input === false ? 'disabled' : 'auto',
...(debugEnabled ? { debug: true } : {}),
yes: options.yes === true,
cliVersion: context.packageInfo.version,
...(options.llmBackend ? { llmBackend: options.llmBackend } : {}),
...(options.anthropicApiKeyEnv ? { anthropicApiKeyEnv: options.anthropicApiKeyEnv } : {}),
...(options.anthropicApiKeyFile ? { anthropicApiKeyFile: options.anthropicApiKeyFile } : {}),
...(options.llmModel ? { llmModel: options.llmModel } : {}),
...(options.vertexProject ? { vertexProject: options.vertexProject } : {}),
...(options.vertexLocation ? { vertexLocation: options.vertexLocation } : {}),
skipLlm: options.skipLlm === true,
@ -501,11 +465,6 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
...(options.metabaseDatabaseId !== undefined ? { metabaseDatabaseId: options.metabaseDatabaseId } : {}),
...(options.notionCrawlMode ? { notionCrawlMode: options.notionCrawlMode } : {}),
...(options.notionRootPageId.length > 0 ? { notionRootPageIds: options.notionRootPageId } : {}),
...(options.gdriveServiceAccountKeyRef
? { gdriveServiceAccountKeyRef: options.gdriveServiceAccountKeyRef }
: {}),
...(options.gdriveFolderId ? { gdriveFolderId: options.gdriveFolderId } : {}),
...(options.gdriveRecursive ? { gdriveRecursive: true } : {}),
runInitialSourceIngest: false,
skipSources: options.skipSources === true,
showEntryMenu: shouldShowSetupEntryMenu(options, command),

View file

@ -44,7 +44,7 @@ export function registerSlCommands(program: Command, context: KtxCliCommandConte
.description('List, search, validate, or query local semantic-layer sources')
.usage('[options] [query...]')
.argument('[query...]', 'Search query; omit to list all sources')
.option('--connection-id <id>', 'ktx connection id')
.option('--connection-id <id>', 'KTX connection id')
.option('--limit <number>', 'Maximum search results (search mode only)', parsePositiveIntegerOption)
.addOption(
new Option('--output <mode>', 'Output mode: pretty (default in TTY), plain (TSV), or json').choices([
@ -94,28 +94,19 @@ export function registerSlCommands(program: Command, context: KtxCliCommandConte
},
);
sl.command('read')
.description('Read a semantic-layer source YAML file')
.argument('<sourceName>', 'Semantic-layer source name')
.action(async (sourceName: string, _options, command) => {
const parentOpts = command.parent?.opts() as { connectionId?: string } | undefined;
await runSlArgs(context, {
command: 'read',
projectDir: resolveCommandProjectDir(command),
connectionId: parentOpts?.connectionId,
sourceName,
});
});
sl.command('validate')
.description('Validate a semantic-layer source')
.description('Validate a semantic-layer source (set --connection-id on `ktx sl`)')
.argument('<sourceName>', 'Semantic-layer source name')
.action(async (sourceName: string, _options, command) => {
const parentOpts = command.parent?.opts() as { connectionId?: string } | undefined;
const connectionId = parentOpts?.connectionId;
if (connectionId === undefined) {
command.error("error: required option '--connection-id <id>' not specified");
}
await runSlArgs(context, {
command: 'validate',
projectDir: resolveCommandProjectDir(command),
connectionId: parentOpts?.connectionId,
connectionId: connectionId as string,
sourceName,
});
});
@ -140,14 +131,10 @@ export function registerSlCommands(program: Command, context: KtxCliCommandConte
throw new Error('sl query requires at least one --measure');
}
const parentOpts = command.parent?.opts() as { connectionId?: string } | undefined;
const connectionId = parentOpts?.connectionId;
if (connectionId === undefined) {
command.error("error: required option '--connection-id <id>' not specified");
}
const args = slQueryCommandSchema.parse({
command: 'query',
projectDir: resolveCommandProjectDir(command),
connectionId,
connectionId: parentOpts?.connectionId,
...(options.queryFile
? { queryFile: options.queryFile }
: {

View file

@ -26,7 +26,7 @@ export function registerSqlCommands(program: Command, context: KtxCliCommandCont
.command('sql')
.description('Execute parser-validated read-only SQL against a configured connection')
.argument('<sql...>', 'SQL query to execute')
.requiredOption('-c, --connection <id>', 'ktx connection id')
.requiredOption('-c, --connection <id>', 'KTX connection id')
.option('--max-rows <n>', 'Maximum rows to return', parseSqlMaxRowsOption, DEFAULT_MAX_ROWS)
.addOption(
new Option('--output <mode>', 'Output mode: pretty (default), plain (TSV), or json').choices([

View file

@ -15,7 +15,7 @@ function inputMode(options: { input?: boolean }): { inputMode?: 'disabled' } {
export function registerStatusCommands(program: Command, context: KtxCliCommandContext): void {
program
.command('status')
.description('Check current ktx setup and project readiness')
.description('Check current KTX setup and project readiness')
.option('--json', 'Print JSON output', false)
.option('-v, --verbose', 'Show every check, including passing ones', false)
.option('--validate', 'Only validate the ktx.yaml schema; skip readiness checks', false)

View file

@ -1,28 +0,0 @@
import type { KtxCliIo } from './cli-runtime.js';
import { isWritableTtyOutput } from './io/tty.js';
import { dim } from './io/symbols.js';
import { SLACK_URL } from './links.js';
type ErrorCtaVariant = 'error' | 'crash';
/** @internal */
export const SLACK_HELP_FOOTER = `Community & support: ${SLACK_URL}`;
/** @internal */
export const SLACK_SETUP_NOTE = {
title: 'Community',
body: `Questions or feedback? Join the ktx Slack: ${SLACK_URL}`,
} as const;
export function writeErrorCommunityHint(io: KtxCliIo, variant: ErrorCtaVariant): void {
if (!isWritableTtyOutput(io.stderr)) {
return;
}
const line =
variant === 'crash'
? `This may be a bug - report it or ask in the ktx community: ${SLACK_URL}`
: `Stuck? The ktx community can help: ${SLACK_URL}`;
io.stderr.write(`${dim(line)}\n`);
}

View file

@ -1,172 +0,0 @@
import type { CommandUnknownOpts, Option } from '@commander-js/extra-typings';
/**
* Dynamic completion candidates that depend on project state (semantic-layer
* source names, wiki page keys, connection ids). Injected so the engine stays
* pure and unit-testable without touching the filesystem.
*/
export interface CompletionProviders {
/** Candidate operands for a positional argument of the active command path. */
positionalCandidates(commandPath: string[], typedTokens: string[]): Promise<string[]>;
/** Candidate values for an option that has no static `choices` (e.g. `--connection-id`). */
optionValueCandidates(commandPath: string[], optionFlag: string, typedTokens: string[]): Promise<string[]>;
}
interface ResolvedCommand {
command: CommandUnknownOpts;
/** Subcommand names from the root down to the active command (root name excluded). */
commandPath: string[];
}
function isHiddenCommand(command: CommandUnknownOpts): boolean {
// Completion mirrors `ktx --help`: commands registered with `{ hidden: true }`
// (the `__complete` helper and `mcp serve-internal`) are internal and must not
// surface. Commander exposes this only through the private `_hidden` field its
// own help renderer reads, so a name heuristic like a `__` prefix is not enough.
return (command as { _hidden?: boolean })._hidden === true;
}
function resolveCommand(program: CommandUnknownOpts, typedTokens: string[]): ResolvedCommand {
let command: CommandUnknownOpts = program;
const commandPath: string[] = [];
for (let index = 0; index < typedTokens.length; index += 1) {
const token = typedTokens[index];
if (token.startsWith('-')) {
// A value-taking option in the `--flag value` form consumes the next token
// as its value, so skip that value before matching subcommands. Otherwise a
// connection id like `query` would be resolved as the `sl query` subcommand
// instead of being treated as the `--connection-id` value. The `--flag=value`
// form carries its own value and consumes nothing extra.
if (!token.includes('=')) {
const option = findOption(command, token);
if (option && !option.isBoolean()) {
index += 1;
}
}
continue;
}
const sub = command.commands.find((candidate) => candidate.name() === token || candidate.aliases().includes(token));
if (sub) {
command = sub;
commandPath.push(sub.name());
}
}
return { command, commandPath };
}
function collectOptions(command: CommandUnknownOpts): Option[] {
const options: Option[] = [];
let current: CommandUnknownOpts | null = command;
while (current) {
options.push(...current.options);
current = current.parent;
}
return options;
}
function findOption(command: CommandUnknownOpts, flag: string): Option | undefined {
return collectOptions(command).find((option) => option.long === flag || option.short === flag);
}
function isRepeatableOption(option: Option): boolean {
// Variadic options, and options backed by a collector with an array default
// (e.g. `--measure`/`--dimension`), may be supplied more than once.
return option.variadic || Array.isArray(option.defaultValue);
}
function flagCandidates(command: CommandUnknownOpts, typedTokens: string[]): string[] {
const present = new Set(typedTokens.filter((token) => token.startsWith('-')));
const candidates: string[] = [];
for (const option of collectOptions(command)) {
if (option.hidden || !option.long) {
continue;
}
if (present.has(option.long) && !isRepeatableOption(option)) {
continue;
}
candidates.push(option.long);
}
return candidates;
}
async function optionValueCandidates(
resolved: ResolvedCommand,
option: Option,
typedTokens: string[],
providers: CompletionProviders,
): Promise<string[]> {
if (option.argChoices && option.argChoices.length > 0) {
return option.argChoices;
}
return providers.optionValueCandidates(resolved.commandPath, option.long ?? option.name(), typedTokens);
}
function dedupeSortFilter(candidates: string[], partial: string): string[] {
const seen = new Set<string>();
const matches: string[] = [];
for (const candidate of candidates) {
if (!candidate.startsWith(partial) || seen.has(candidate)) {
continue;
}
seen.add(candidate);
matches.push(candidate);
}
return matches.sort();
}
/**
* Compute completion candidates for the partial last element of `words`
* (everything the shell has on the line after `ktx`). The active command and
* its flags are derived by walking the live Commander tree, so completion never
* drifts from the real command structure.
*/
export async function computeCompletions(
program: CommandUnknownOpts,
words: string[],
providers: CompletionProviders,
): Promise<string[]> {
const partial = words.length > 0 ? (words[words.length - 1] ?? '') : '';
const typedTokens = words.slice(0, -1);
const resolved = resolveCommand(program, typedTokens);
// (a) Option value via the `--opt=value` form.
const equalsMatch = /^(--[^=]+)=(.*)$/.exec(partial);
if (equalsMatch) {
const [, flag, valuePartial] = equalsMatch;
const option = findOption(resolved.command, flag);
if (!option || option.isBoolean()) {
return [];
}
const values = await optionValueCandidates(resolved, option, typedTokens, providers);
return dedupeSortFilter(
values.map((value) => `${flag}=${value}`),
`${flag}=${valuePartial}`,
);
}
// (b) Option value via the `--opt value` form (previous token is a value-taking option).
const previous = typedTokens[typedTokens.length - 1];
if (previous && previous.startsWith('-') && !partial.startsWith('-')) {
const option = findOption(resolved.command, previous);
if (option && !option.isBoolean()) {
return dedupeSortFilter(await optionValueCandidates(resolved, option, typedTokens, providers), partial);
}
}
// (c) Flag completion.
if (partial.startsWith('-')) {
return dedupeSortFilter(flagCandidates(resolved.command, typedTokens), partial);
}
// (d) Positional: subcommand names union static argument choices union dynamic operand candidates.
const candidates: string[] = resolved.command.commands
.filter((sub) => !isHiddenCommand(sub))
.map((sub) => sub.name());
for (const argument of resolved.command.registeredArguments) {
if (argument.argChoices) {
candidates.push(...argument.argChoices);
}
}
candidates.push(...(await providers.positionalCandidates(resolved.commandPath, typedTokens)));
return dedupeSortFilter(candidates, partial);
}

Some files were not shown because too many files have changed in this diff Show more