mirror of
https://github.com/Kaelio/ktx.git
synced 2026-06-22 08:38:08 +02:00
Merge origin/main into add-ktx-mcp-claude-desktop
This commit is contained in:
Andrey Avtomonov
commit
af4f2c29df
25 changed files with 1657 additions and 234 deletions
223
README.md
223
README.md
|
|
@ -2,9 +2,9 @@
|
|||
<img src="assets/ktx-lockup.svg" alt="KTX" width="500" />
|
||||
</h1>
|
||||
|
||||
<p align="center">
|
||||
<strong>The context layer for analytics agents</strong>
|
||||
</p>
|
||||
<h1 align="center">
|
||||
The context layer for analytics agents
|
||||
</h1>
|
||||
|
||||
<p align="center">by Kaelio</p>
|
||||
|
||||
|
|
@ -18,157 +18,80 @@
|
|||
---
|
||||
|
||||
KTX turns warehouse metadata, semantic definitions, and business knowledge into
|
||||
reviewable project files that agents can use while planning, querying, and
|
||||
updating analytics work.
|
||||
|
||||
A KTX project is a directory of plain files - YAML semantic sources, Markdown
|
||||
wiki pages, and SQLite state - that you commit to git and review in PRs,
|
||||
just like dbt models.
|
||||
|
||||
## Who KTX is for
|
||||
|
||||
KTX is built for analytics engineers and data teams who want data agents to
|
||||
work on real analytics systems - not just generate one-off SQL.
|
||||
reviewable project files that agents can use to plan, query, and update
|
||||
analytics work.
|
||||
|
||||
Use KTX when you want agents to:
|
||||
|
||||
- **Generate SQL** from approved measures and joins
|
||||
- **Repair semantic definitions** through reviewable diffs
|
||||
- **Explain metric provenance** with warehouse evidence
|
||||
- **Work alongside** dbt, LookML, MetricFlow, Looker, Metabase, and modern BI
|
||||
platforms
|
||||
- Generate SQL from approved measures and joins
|
||||
- Repair semantic definitions through reviewable diffs
|
||||
- Explain metric provenance with warehouse evidence
|
||||
- Work alongside dbt, MetricFlow, LookML, Looker, Metabase, and Notion
|
||||
|
||||
Works with PostgreSQL, Snowflake, BigQuery, ClickHouse, MySQL, SQL Server, and
|
||||
Supports PostgreSQL, Snowflake, BigQuery, ClickHouse, MySQL, SQL Server, and
|
||||
SQLite.
|
||||
|
||||
## Quick start
|
||||
|
||||
Install the CLI and run the setup wizard:
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
npm install @kaelio/ktx
|
||||
npm install -g @kaelio/ktx
|
||||
pnpm add --global @kaelio/ktx
|
||||
ktx setup
|
||||
```
|
||||
|
||||
The wizard walks through six steps: configuring your LLM provider, setting up
|
||||
embeddings, connecting your database, adding context sources (dbt, LookML,
|
||||
Metabase, Looker, Notion), building context, and installing agent integration.
|
||||
|
||||
If it exits before completion, rerun `ktx setup` to resume where you left off.
|
||||
|
||||
Check your project status:
|
||||
|
||||
```bash
|
||||
ktx status
|
||||
```
|
||||
|
||||
```
|
||||
KTX project: /home/user/analytics
|
||||
Project ready: yes
|
||||
LLM ready: yes (claude-sonnet-4-6)
|
||||
Embeddings ready: yes (text-embedding-3-small)
|
||||
Databases configured: yes (postgres-warehouse)
|
||||
Context sources configured: yes (dbt-main)
|
||||
KTX context built: yes
|
||||
Agent integration ready: yes (claude-code:project)
|
||||
```
|
||||
`ktx setup` creates or resumes a local KTX project, configures providers and
|
||||
connections, builds context, and installs agent integration.
|
||||
|
||||
Generate SQL from a semantic-layer source:
|
||||
## Common Commands
|
||||
|
||||
```bash
|
||||
npx @kaelio/ktx sl query --project-dir "$PROJECT_DIR" \
|
||||
--connection-id warehouse \
|
||||
--measure accounts.account_count \
|
||||
--dimension accounts.segment \
|
||||
--format sql
|
||||
```
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `ktx setup` | Create, resume, or update a KTX project |
|
||||
| `ktx status` | Check project readiness |
|
||||
| `ktx connection list` | List configured connections |
|
||||
| `ktx connection test <id>` | Test one connection |
|
||||
| `ktx ingest <id>` | Build context for one connection |
|
||||
| `ktx ingest --all` | Build context for every configured connection |
|
||||
| `ktx ingest text <file>` | Capture free-form notes into memory |
|
||||
| `ktx sl list` | List semantic-layer sources |
|
||||
| `ktx sl search "revenue"` | Search semantic-layer sources |
|
||||
| `ktx sl validate <source> --connection-id <id>` | Validate a semantic source |
|
||||
| `ktx sl query --measure <measure> --format sql` | Compile semantic-layer SQL |
|
||||
| `ktx wiki search "revenue definition"` | Search local wiki context |
|
||||
| `ktx mcp start` | Start the local MCP server for agent clients |
|
||||
|
||||
List and test a configured warehouse connection:
|
||||
Project resolution defaults to `KTX_PROJECT_DIR`, then the nearest `ktx.yaml`,
|
||||
then the current directory. Pass `--project-dir <path>` when scripting.
|
||||
|
||||
```bash
|
||||
ktx connection list --project-dir "$PROJECT_DIR"
|
||||
ktx connection test warehouse --project-dir "$PROJECT_DIR"
|
||||
```
|
||||
|
||||
The connection test prints the configured driver and connector-specific status:
|
||||
## Project Layout
|
||||
|
||||
```text
|
||||
Connection test passed: warehouse
|
||||
Driver: sqlite
|
||||
Status: ok
|
||||
```
|
||||
|
||||
## What's in a project
|
||||
|
||||
```
|
||||
my-project/
|
||||
├── ktx.yaml # Project configuration
|
||||
├── semantic-layer/
|
||||
│ └── warehouse/
|
||||
│ ├── orders.yaml # Semantic source definitions
|
||||
│ ├── customers.yaml
|
||||
│ └── order_items.yaml
|
||||
├── wiki/
|
||||
│ ├── global/
|
||||
│ │ ├── revenue.md # Business definitions and rules
|
||||
│ │ └── segment-classification.md
|
||||
│ └── user/
|
||||
│ └── local/
|
||||
├── raw-sources/
|
||||
│ └── warehouse/
|
||||
│ └── <syncId>/ # Database ingest artifacts and reports
|
||||
└── .ktx/
|
||||
└── db.sqlite # Local state (git-ignored)
|
||||
├── ktx.yaml # Project configuration
|
||||
├── semantic-layer/<connection-id>/ # YAML semantic sources
|
||||
├── wiki/global/ # Shared business context
|
||||
├── wiki/user/<user-id>/ # User-scoped notes
|
||||
├── raw-sources/<connection-id>/ # Ingest artifacts and reports
|
||||
└── .ktx/ # Local state and secrets, git-ignored
|
||||
```
|
||||
|
||||
Semantic sources and wiki pages are committed to git. The `.ktx/` directory
|
||||
holds ephemeral state and is git-ignored - delete it and KTX rebuilds on the
|
||||
next run.
|
||||
Commit `ktx.yaml`, `semantic-layer/`, and `wiki/`. Keep `.ktx/` local.
|
||||
|
||||
### Build demo warehouse context
|
||||
## Agent Usage
|
||||
|
||||
Database ingest artifacts are written under `raw-sources/warehouse/<syncId>/`
|
||||
in the project directory.
|
||||
Setup can install KTX instructions for Claude Code, Codex, Cursor, OpenCode,
|
||||
and universal `.agents` clients:
|
||||
|
||||
```bash
|
||||
ktx ingest warehouse --project-dir "$PROJECT_DIR" --fast
|
||||
ktx status --project-dir "$PROJECT_DIR"
|
||||
ktx setup --agents --target codex
|
||||
```
|
||||
|
||||
For non-SQLite drivers, prefer credential references such as `--url env:NAME`
|
||||
or `--url file:PATH` over literal credential URLs.
|
||||
|
||||
## Managed Python runtime
|
||||
|
||||
KTX installs its Python runtime only when a Python-backed command needs it.
|
||||
The runtime lives outside the npm cache, is versioned by the installed CLI
|
||||
version, and is managed by `ktx dev runtime` commands.
|
||||
|
||||
KTX requires `uv` on `PATH` to create the managed runtime. Install `uv` with
|
||||
your system package manager or the official installer before running Python-
|
||||
backed KTX commands. KTX doesn't download `uv` automatically; run
|
||||
`ktx dev runtime status` if runtime installation fails:
|
||||
Agent-facing workflows typically start with:
|
||||
|
||||
```bash
|
||||
ktx dev runtime install --yes
|
||||
ktx dev runtime status
|
||||
ktx dev runtime start
|
||||
ktx dev runtime stop
|
||||
```
|
||||
|
||||
The release artifact manifest contains the public npm tarball and the bundled `kaelio-ktx`
|
||||
runtime wheel. The `python/ktx-sl` and `python/ktx-daemon` directories remain
|
||||
source packages for development, not public release artifacts.
|
||||
|
||||
## Use KTX with agents
|
||||
|
||||
KTX exposes context to end-user agents through MCP tools. The CLI remains the
|
||||
admin surface for setup, ingest, status, daemon lifecycle, and debugging.
|
||||
|
||||
```bash
|
||||
ktx mcp start
|
||||
ktx setup --agents
|
||||
ktx sl search "revenue" --json
|
||||
ktx wiki search "refund policy" --json
|
||||
ktx sl query --connection-id warehouse --measure orders.revenue --format sql
|
||||
```
|
||||
|
||||
During agent setup, choose **MCP tools + analytics skill** for client agents.
|
||||
|
|
@ -178,8 +101,7 @@ or operator agent also needs pinned `ktx` admin commands.
|
|||
The analytics skill teaches client agents the MCP workflow: discover data,
|
||||
prefer semantic-layer measures, inspect entity details before raw SQL, and
|
||||
capture durable learnings. Admin CLI skills call `ktx` commands directly
|
||||
through a skill file installed in your agent's config, for example
|
||||
`.claude/skills/ktx/SKILL.md`:
|
||||
through a skill file installed in your agent's config:
|
||||
|
||||
```bash
|
||||
ktx sl query --measure orders.revenue --dimension orders.status --format sql
|
||||
|
|
@ -189,9 +111,14 @@ ktx sl validate orders
|
|||
|
||||
Supported client agents: Claude Code, Claude Desktop, Codex, Cursor, OpenCode,
|
||||
and clients that can use the printed MCP endpoint or `.agents` admin skills.
|
||||
Claude Desktop setup generates `.ktx/agents/claude/ktx-plugin.zip`, which
|
||||
bundles the analytics skill and a local `ktx mcp stdio` server config for
|
||||
Claude Desktop's plugin installer.
|
||||
Claude Desktop setup registers a local `ktx mcp stdio` server in Claude
|
||||
Desktop's config and generates `.ktx/agents/claude/ktx-plugin.zip` with the
|
||||
analytics skill.
|
||||
|
||||
The release artifact manifest contains the public npm tarball and the bundled
|
||||
`kaelio-ktx` runtime wheel. The `python/ktx-sl` and `python/ktx-daemon`
|
||||
directories remain source packages for development, not public release
|
||||
artifacts.
|
||||
|
||||
## Workspace packages
|
||||
|
||||
|
|
@ -221,7 +148,7 @@ pnpm run build
|
|||
pnpm run check
|
||||
```
|
||||
|
||||
Use the development CLI for local testing:
|
||||
Use the development CLI locally:
|
||||
|
||||
```bash
|
||||
pnpm run setup:dev
|
||||
|
|
@ -229,30 +156,28 @@ pnpm run link:dev
|
|||
ktx-dev --help
|
||||
```
|
||||
|
||||
### Debug LLM traces
|
||||
KTX is a pnpm + uv workspace:
|
||||
|
||||
KTX can capture local AI SDK DevTools traces for LLM calls that run through the
|
||||
KTX provider. Enable it with an environment flag when running an LLM-backed
|
||||
command:
|
||||
- TypeScript packages live in `packages/*`
|
||||
- CLI source lives in `packages/cli`
|
||||
- Python runtime source lives in `python/ktx-sl` and `python/ktx-daemon`
|
||||
- Public docs live in `docs-site/content/docs`
|
||||
|
||||
Useful checks:
|
||||
|
||||
```bash
|
||||
KTX_AI_DEVTOOLS_ENABLED=true ktx ingest warehouse --project-dir "$PROJECT_DIR" --deep
|
||||
pnpm run type-check
|
||||
pnpm run test
|
||||
pnpm run dead-code
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Traces are written to `.devtools/generations.json` under the current working
|
||||
directory. To inspect them, run:
|
||||
## Docs
|
||||
|
||||
```bash
|
||||
pnpm dlx @ai-sdk/devtools
|
||||
```
|
||||
|
||||
Then open `http://localhost:4983`. These traces are local-development-only and
|
||||
store prompts, model outputs, tool arguments/results, and raw provider payloads
|
||||
in plain text. Do not enable this in production or for sensitive runs.
|
||||
|
||||
The repository uses `pnpm` for TypeScript packages and `uv` for Python
|
||||
packages. See [Contributing](docs-site/content/docs/community/contributing.mdx)
|
||||
for full development setup, testing, and PR guidelines.
|
||||
- [Quickstart](docs-site/content/docs/getting-started/quickstart.mdx)
|
||||
- [CLI Reference](docs-site/content/docs/cli-reference/index.mdx)
|
||||
- [Building Context](docs-site/content/docs/guides/building-context.mdx)
|
||||
- [Contributing](docs-site/content/docs/community/contributing.mdx)
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue