mirror of
https://github.com/Kaelio/ktx.git
synced 2026-06-07 07:55:13 +02:00
* docs: add CLI component reuse guidance * docs: add unified ingest ux design * Refine unified ingest UX design after adversarial review iteration 1 * Refine unified ingest UX design after adversarial review iteration 2 * Refine unified ingest UX design after adversarial review iteration 3 * feat(cli): route public connection ingest command * feat(cli): hide standalone scan from public help * feat(cli): plan public ingest depth and query history * feat(cli): execute public database ingest facets * feat(ingest): read connection query history config * fix(cli): use public ingest wording * fix(config): stop generating ingest adapter allow lists * docs: document public ingest command * test: align ingest surface expectations * docs: add unified ingest public CLI surface plan * feat(cli): preflight deep public ingest readiness * feat(setup): store query history in connection context * feat(setup): store database context depth * feat(setup): verify context readiness by database depth * fix(setup): keep context build foreground only * fix(config): reject reserved ingest connection ids * test: close unified ingest v1 expectations * docs: add unified ingest v1 closure plan * fix(ingest): bypass adapter allow-list for public source ingest * fix(ingest): honor query history window intent * fix(ingest): hide scan internals from public database ingest * feat(ingest): use foreground view for interactive public ingest * fix(setup): use schema context and query history wording * test(cli): verify unified ingest public output * docs: add unified ingest v1 public output closure plan * fix(setup): forward query history flags * fix(setup): prompt for postgres query history * fix(status): report query history readiness * fix(ingest): remove legacy public guidance * fix(ingest): polish foreground retry copy * docs(examples): use unified query history wording * chore(ingest): finish public query history cleanup * docs: add unified ingest v1 query history status cleanup plan * test(docs): cover unified ingest public docs * docs: align ingest CLI reference with unified UX * docs: update context build guides for unified ingest * docs: update setup and primary source ingest wording * docs: stop advertising adapter-backed example ingest * docs: close unified ingest public docs gaps * docs: add unified ingest v1 docs site closure plan * fix: render unified ingest foreground warnings * fix: explain query history schema order * fix: add public ingest retry guidance * fix: align setup next steps with unified ingest * fix: remove scan wording from demo progress * test: verify unified ingest ux closure * docs: add unified ingest v1 foreground and retry closure plan * fix(cli): preserve query-history pull config in public ingest * fix(cli): omit hidden commands from docs command tree * test(cli): close unified ingest final public surface checks * docs: add unified ingest v1 final public surface closure plan * fix(cli): use public source labels in ingest reports * fix(cli): suppress low-level public ingest output * test(cli): verify unified ingest public plain output * docs: add unified ingest v1 public plain output closure plan * fix(cli): add public ingest copy sanitizers * fix(cli): sanitize public ingest progress copy * fix(cli): rename setup schema scope prompt * docs(plan): add progress copy closure; test: align setup back-nav fixture Adds the iter9 plan and updates the setup back-navigation test fixture to pass disableQueryHistory plus listSchemas/listTables stubs that the unified ingest setup step now requires. * docs(plan): add final ux labels plan with narrowed label scans * fix(cli): aggregate unsupported query-history warnings * fix(cli): align setup database labels * test(cli): fix setup database test type-check * fix(cli): remove primary-source wording from setup output * test(cli): verify unified ingest setup closure * docs(plan): add unified ingest v1 verification copy closure plan * fix(cli): remove top-level scan command * fix(cli): remove legacy ingest and wiki commands * Merge scan into ingest flow * feat(cli): split ingest progress into per-phase rows, rename work units to tasks Each database target in the unified ingest dashboard now renders one row per real subprocess (Schema, then Query history when enabled) instead of a single combined bar. Each phase has its own monotonic 0-100% bar so the progress never snaps back to zero when historic-sql starts after scan completes. Completed phases keep their final bar, summary, and elapsed time visible as an inline audit trail; queued and skipped phases are shown explicitly. Also rename user-facing "work units" / "Failed work units" to "tasks" / "Failed tasks" in ingest output and parseIngestSummary. The parser still accepts the legacy "Work units:" wording in captured output for backward compat. Internal memory-flow event names and type fields are left alone. * Fix test harness failures * Fix CI smoke checks --------- Co-authored-by: Andrey Avtomonov <7889985+andreybavt@users.noreply.github.com>
254 lines
9.8 KiB
Text
254 lines
9.8 KiB
Text
---
|
|
title: Quickstart
|
|
description: Set up KTX and build your first context in under 10 minutes.
|
|
---
|
|
|
|
This guide walks you through `ktx setup` — an interactive wizard that configures your LLM provider, connects your database, optionally ingests from your existing tools, builds context, and installs agent integration.
|
|
|
|
If you are a coding assistant trying to decide which KTX docs page to read, start with the [Agent Quickstart](/docs/ai-resources/agent-quickstart). This page is the human setup walkthrough.
|
|
|
|
## Workflow summary
|
|
|
|
Use this sequence when you are setting up KTX in an analytics project:
|
|
|
|
1. `npm install -g @kaelio/ktx` — install the published KTX CLI from npm.
|
|
2. `ktx setup` — create or resume a KTX project.
|
|
|
|
The setup wizard is stateful. If it exits before completion, rerun `ktx setup` in the same project directory to resume from the first incomplete step.
|
|
|
|
## Install and run setup
|
|
|
|
Install the published [`@kaelio/ktx`](https://www.npmjs.com/package/@kaelio/ktx) CLI:
|
|
|
|
```bash
|
|
npm install -g @kaelio/ktx
|
|
```
|
|
|
|
Then run the setup wizard:
|
|
|
|
```bash
|
|
ktx setup
|
|
```
|
|
|
|
The local checkout flow is only for contributors working on KTX itself. See [Contributing](/docs/community/contributing) for that setup.
|
|
|
|
The wizard walks through six steps. You can go back at any point, and if you exit early, rerunning `ktx setup` resumes where you left off.
|
|
|
|
## Step 1: Configure LLM
|
|
|
|
KTX uses an Anthropic model to enrich schema descriptions, generate semantic sources during ingestion, and reconcile metadata from your tools.
|
|
|
|
The wizard asks how to find your API key:
|
|
|
|
```
|
|
◆ How should KTX find your Anthropic API key?
|
|
│ ○ Use ANTHROPIC_API_KEY from the environment
|
|
│ ○ Paste a key and save it as a local secret file
|
|
```
|
|
|
|
If you choose to paste a key, KTX saves it in `.ktx/secrets/anthropic-api-key` with local file permissions. Your `ktx.yaml` stores a `file:` reference, never the raw key.
|
|
|
|
Next, choose a model:
|
|
|
|
```
|
|
◆ Which Anthropic model should KTX use?
|
|
│ ○ Claude Sonnet 4.6 (recommended)
|
|
│ ○ Claude Opus 4.6
|
|
│ ○ Claude Haiku 4.5
|
|
│ ○ Enter a model ID manually
|
|
```
|
|
|
|
KTX runs a health check to verify your key and model work before saving.
|
|
|
|
## Step 2: Configure embeddings
|
|
|
|
KTX uses embeddings for semantic search over sources, wiki content, schema metadata, and relationship evidence.
|
|
|
|
```
|
|
◆ Which embedding option should KTX use?
|
|
│ ○ Local sentence-transformers embeddings
|
|
│ ○ OpenAI embeddings (recommended)
|
|
```
|
|
|
|
**OpenAI embeddings** use `text-embedding-3-small` (1536 dimensions) and require an `OPENAI_API_KEY`.
|
|
|
|
**Local embeddings** use `all-MiniLM-L6-v2` (384 dimensions) via the KTX managed Python runtime. No API key is needed. KTX can install and start the runtime during setup; to prepare it ahead of time, run:
|
|
|
|
```bash
|
|
ktx dev runtime install --feature local-embeddings --yes
|
|
ktx dev runtime start --feature local-embeddings
|
|
```
|
|
|
|
## Step 3: Connect a database
|
|
|
|
Select one or more databases for KTX to connect to. The wizard supports
|
|
SQLite, PostgreSQL, MySQL, ClickHouse, SQL Server, BigQuery, and Snowflake.
|
|
|
|
For PostgreSQL, you can enter connection details field by field or paste a connection URL:
|
|
|
|
```
|
|
◆ How do you want to connect to PostgreSQL?
|
|
│ ○ Enter connection details (host, port, database, user)
|
|
│ ○ Paste a connection URL
|
|
```
|
|
|
|
If your URL contains credentials, KTX saves it to `.ktx/secrets/` and writes a `file:` reference in `ktx.yaml`. You can also use `env:DATABASE_URL` to reference an environment variable.
|
|
|
|
After connecting, KTX automatically runs a connection test and builds fast
|
|
schema context:
|
|
|
|
```
|
|
Testing postgres-warehouse
|
|
Connection test passed
|
|
Driver: PostgreSQL - Tables: 42
|
|
|
|
Building schema context for postgres-warehouse
|
|
Running fast database ingest
|
|
|
|
Schema context complete for postgres-warehouse
|
|
Changes: 42 new tables
|
|
|
|
Database ready
|
|
postgres-warehouse - PostgreSQL - schema context complete
|
|
```
|
|
|
|
For PostgreSQL, Snowflake, and BigQuery, the wizard can enable query-history
|
|
ingest when the warehouse history feature is available. Query history is stored
|
|
under `connections.<id>.context.queryHistory` in `ktx.yaml`.
|
|
|
|
## Step 4: Add context sources
|
|
|
|
Context sources let KTX ingest metadata from your existing analytics tools. This step is optional — you can skip it and add sources later.
|
|
|
|
```
|
|
◆ Which context sources should KTX ingest?
|
|
│ ◻ dbt
|
|
│ ◻ MetricFlow
|
|
│ ◻ Metabase
|
|
│ ◻ Looker
|
|
│ ◻ LookML
|
|
│ ◻ Notion
|
|
```
|
|
|
|
For **dbt**, point KTX at a local path or git URL. KTX reads your `dbt_project.yml` and schema files to extract model metadata:
|
|
|
|
```
|
|
◆ dbt source location
|
|
│ ○ Local path
|
|
│ ○ Git URL
|
|
```
|
|
|
|
For **Metabase** and **Looker**, you provide an API URL and credentials. KTX maps BI databases to your KTX primary source connections so it knows which warehouse tables the BI metadata refers to.
|
|
|
|
Context sources are saved to `ktx.yaml` and built during the next step.
|
|
|
|
## Step 5: Build context
|
|
|
|
This is where KTX builds agent-ready context. It uses the database context
|
|
depth saved by setup and ingests metadata from any configured context sources.
|
|
|
|
```
|
|
◆ Build KTX context for agents?
|
|
│ ○ Build context now (recommended)
|
|
│ ○ Leave context unbuilt and exit setup
|
|
```
|
|
|
|
Fast database context builds deterministic schema grounding. Deep database
|
|
context also generates AI descriptions, embeddings, and relationship evidence
|
|
when those capabilities are configured.
|
|
|
|
For a small database (under 50 tables), this can take a few minutes. Larger
|
|
warehouses can take longer. Context builds run in the foreground; press
|
|
<kbd>Ctrl+C</kbd> to stop the current run and rerun `ktx setup` or `ktx ingest`
|
|
when you are ready to try again.
|
|
|
|
When the build completes, KTX verifies that agent-ready context was produced:
|
|
|
|
```
|
|
KTX context is ready for agents.
|
|
|
|
Databases:
|
|
postgres-warehouse: deep context complete
|
|
|
|
Context sources:
|
|
dbt-main: memory update complete
|
|
|
|
Verification:
|
|
Agent context: ready
|
|
Semantic search: ready
|
|
```
|
|
|
|
## Step 6: Install agent integration
|
|
|
|
The final step connects KTX to your coding agent. Choose how agents should access the project:
|
|
|
|
```
|
|
◆ How should agents use this KTX project?
|
|
│ ○ CLI tools and skills
|
|
```
|
|
|
|
Then select which agents to install for:
|
|
|
|
```
|
|
◆ Which agent targets should KTX install?
|
|
│ ◻ Claude Code
|
|
│ ◻ Codex
|
|
│ ◻ Cursor
|
|
│ ◻ OpenCode
|
|
│ ◻ Custom agent (.agents)
|
|
```
|
|
|
|
**CLI mode** writes a skill file (e.g., `.claude/skills/ktx/SKILL.md`) that teaches the agent to call KTX commands directly.
|
|
|
|
**Custom agent** uses the universal `.agents` target for agents that can read project-local skills.
|
|
|
|
## Generated files
|
|
|
|
KTX writes project state as plain files so agents can inspect and edit changes in git.
|
|
|
|
| Path | Created by | Purpose |
|
|
|------|------------|---------|
|
|
| `ktx.yaml` | `ktx setup` | Main project configuration: connections, LLM settings, embeddings, and context sources |
|
|
| `.ktx/secrets/*` | `ktx setup` when file-backed secrets are selected | Local secret files referenced from `ktx.yaml`; do not commit these |
|
|
| `semantic-layer/<connection-id>/*.yaml` | context build, ingestion, or direct file edits | Semantic source definitions agents use for SQL generation |
|
|
| `wiki/global/*.md` | ingestion, memory capture, or direct file edits | Shared business context and metric definitions |
|
|
| `wiki/user/<user-id>/*.md` | memory capture or direct file edits | User-scoped notes for one agent/user context |
|
|
| `.claude/skills/ktx/SKILL.md`, `.agents/skills/ktx/SKILL.md` | CLI-mode agent integration setup | Agent instructions for calling public `ktx` commands |
|
|
|
|
## Verify it worked
|
|
|
|
Check your project status:
|
|
|
|
```bash
|
|
ktx status
|
|
```
|
|
|
|
```
|
|
KTX project: /home/user/analytics
|
|
Project ready: yes
|
|
LLM ready: yes (claude-sonnet-4-6)
|
|
Embeddings ready: yes (text-embedding-3-small)
|
|
Databases configured: yes (postgres-warehouse)
|
|
Context sources configured: yes (dbt-main)
|
|
KTX context built: yes
|
|
Agent integration ready: yes (claude-code:project)
|
|
```
|
|
|
|
## Common errors
|
|
|
|
| Error or symptom | Likely cause | Recovery |
|
|
|------------------|--------------|----------|
|
|
| `ktx: command not found` | The KTX package is not installed globally, or the shell cannot find the global binary | Run `npm install -g @kaelio/ktx` and open a new shell |
|
|
| LLM health check fails | Missing, invalid, or unauthorized Anthropic API key | Export `ANTHROPIC_API_KEY` or rerun `ktx setup` and choose the file-backed secret option |
|
|
| OpenAI embedding check fails | `OPENAI_API_KEY` is missing when OpenAI embeddings are selected | Export `OPENAI_API_KEY`, or rerun setup and choose local sentence-transformers embeddings |
|
|
| Local embeddings hang or fail | The managed Python runtime cannot start or the local model runtime is unavailable | Install `uv`, run `ktx dev runtime status`, then run `ktx dev runtime install --feature local-embeddings --yes` and rerun setup |
|
|
| Database connection test fails | Credentials, network access, warehouse, database, or schema value is wrong | Test the same URL with the database's native client, then rerun `ktx setup` and reconfigure the connection |
|
|
| `KTX context built: no` in `ktx status` | Setup saved configuration but did not build context | Run `ktx setup` and choose to build context now |
|
|
| Agent integration is incomplete | Setup skipped the agents step or the target was not installed | Run `ktx setup --agents --target codex` using the target you need |
|
|
|
|
## Next steps
|
|
|
|
- **Build more context** — learn about [database ingest](/docs/guides/building-context), relationship detection, and source ingestion workflows in the Building Context guide.
|
|
- **Refine your semantic layer** — the [Writing Context](/docs/guides/writing-context) guide covers source YAML, measures, joins, and wiki pages.
|
|
- **Understand the architecture** — read [The Context Layer](/docs/concepts/the-context-layer) to learn why a context layer is more than a semantic layer.
|
|
- **Connect more agents** — see the [Agent Clients](/docs/integrations/agent-clients) integration page for per-tool setup details.
|