mirror of
https://github.com/Kaelio/ktx.git
synced 2026-06-13 08:15:14 +02:00
docs: simplify readme
This commit is contained in:
Luca Martial
parent
50ffebd98b
commit
cf83cc8acb
1 changed files with 65 additions and 168 deletions
233
README.md
233
README.md
|
|
@ -2,9 +2,9 @@
|
|||
<img src="assets/ktx-lockup.svg" alt="KTX" width="500" />
|
||||
</h1>
|
||||
|
||||
<p align="center">
|
||||
<strong>The context layer for analytics agents</strong>
|
||||
</p>
|
||||
<h2 align="center">
|
||||
The context layer for analytics agents
|
||||
</h2>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://www.npmjs.com/package/@kaelio/ktx"><img src="https://img.shields.io/npm/v/@kaelio/ktx?style=flat-square&color=f97316" alt="npm version" /></a>
|
||||
|
|
@ -16,183 +16,82 @@
|
|||
---
|
||||
|
||||
KTX turns warehouse metadata, semantic definitions, and business knowledge into
|
||||
reviewable project files that agents can use while planning, querying, and
|
||||
updating analytics work.
|
||||
|
||||
A KTX project is a directory of plain files - YAML semantic sources, Markdown
|
||||
wiki pages, and SQLite state - that you commit to git and review in PRs,
|
||||
just like dbt models.
|
||||
|
||||
## Who KTX is for
|
||||
|
||||
KTX is built for analytics engineers and data teams who want data agents to
|
||||
work on real analytics systems - not just generate one-off SQL.
|
||||
reviewable project files that agents can use to plan, query, and update
|
||||
analytics work.
|
||||
|
||||
Use KTX when you want agents to:
|
||||
|
||||
- **Generate SQL** from approved measures and joins
|
||||
- **Repair semantic definitions** through reviewable diffs
|
||||
- **Explain metric provenance** with warehouse evidence
|
||||
- **Work alongside** dbt, LookML, MetricFlow, Looker, Metabase, and modern BI
|
||||
platforms
|
||||
- Generate SQL from approved measures and joins
|
||||
- Repair semantic definitions through reviewable diffs
|
||||
- Explain metric provenance with warehouse evidence
|
||||
- Work alongside dbt, MetricFlow, LookML, Looker, Metabase, and Notion
|
||||
|
||||
Works with PostgreSQL, Snowflake, BigQuery, ClickHouse, MySQL, SQL Server, and
|
||||
Supports PostgreSQL, Snowflake, BigQuery, ClickHouse, MySQL, SQL Server, and
|
||||
SQLite.
|
||||
|
||||
## Quick start
|
||||
|
||||
Install the CLI and run the setup wizard:
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
npm install @kaelio/ktx
|
||||
npm install -g @kaelio/ktx
|
||||
pnpm add --global @kaelio/ktx
|
||||
ktx setup
|
||||
```
|
||||
|
||||
The wizard walks through six steps: configuring your LLM provider, setting up
|
||||
embeddings, connecting your database, adding context sources (dbt, LookML,
|
||||
Metabase, Looker, Notion), building context, and installing agent integration.
|
||||
|
||||
If it exits before completion, rerun `ktx setup` to resume where you left off.
|
||||
|
||||
Check your project status:
|
||||
|
||||
```bash
|
||||
ktx status
|
||||
```
|
||||
|
||||
```
|
||||
KTX project: /home/user/analytics
|
||||
Project ready: yes
|
||||
LLM ready: yes (claude-sonnet-4-6)
|
||||
Embeddings ready: yes (text-embedding-3-small)
|
||||
Databases configured: yes (postgres-warehouse)
|
||||
Context sources configured: yes (dbt-main)
|
||||
KTX context built: yes
|
||||
Agent integration ready: yes (claude-code:project)
|
||||
```
|
||||
`ktx setup` creates or resumes a local KTX project, configures providers and
|
||||
connections, builds context, and installs agent integration.
|
||||
|
||||
Generate SQL from a semantic-layer source:
|
||||
## Common Commands
|
||||
|
||||
```bash
|
||||
npx @kaelio/ktx sl query --project-dir "$PROJECT_DIR" \
|
||||
--connection-id warehouse \
|
||||
--measure accounts.account_count \
|
||||
--dimension accounts.segment \
|
||||
--format sql
|
||||
```
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `ktx setup` | Create, resume, or update a KTX project |
|
||||
| `ktx status` | Check project readiness |
|
||||
| `ktx connection list` | List configured connections |
|
||||
| `ktx connection test <id>` | Test one connection |
|
||||
| `ktx ingest <id>` | Build context for one connection |
|
||||
| `ktx ingest --all` | Build context for every configured connection |
|
||||
| `ktx ingest text <file>` | Capture free-form notes into memory |
|
||||
| `ktx sl list` | List semantic-layer sources |
|
||||
| `ktx sl search "revenue"` | Search semantic-layer sources |
|
||||
| `ktx sl validate <source> --connection-id <id>` | Validate a semantic source |
|
||||
| `ktx sl query --measure <measure> --format sql` | Compile semantic-layer SQL |
|
||||
| `ktx wiki search "revenue definition"` | Search local wiki context |
|
||||
| `ktx mcp start` | Start the local MCP server for agent clients |
|
||||
|
||||
List and test a configured warehouse connection:
|
||||
Project resolution defaults to `KTX_PROJECT_DIR`, then the nearest `ktx.yaml`,
|
||||
then the current directory. Pass `--project-dir <path>` when scripting.
|
||||
|
||||
```bash
|
||||
ktx connection list --project-dir "$PROJECT_DIR"
|
||||
ktx connection test warehouse --project-dir "$PROJECT_DIR"
|
||||
```
|
||||
|
||||
The connection test prints the configured driver and connector-specific status:
|
||||
## Project Layout
|
||||
|
||||
```text
|
||||
Connection test passed: warehouse
|
||||
Driver: sqlite
|
||||
Status: ok
|
||||
```
|
||||
|
||||
## What's in a project
|
||||
|
||||
```
|
||||
my-project/
|
||||
├── ktx.yaml # Project configuration
|
||||
├── semantic-layer/
|
||||
│ └── warehouse/
|
||||
│ ├── orders.yaml # Semantic source definitions
|
||||
│ ├── customers.yaml
|
||||
│ └── order_items.yaml
|
||||
├── wiki/
|
||||
│ ├── global/
|
||||
│ │ ├── revenue.md # Business definitions and rules
|
||||
│ │ └── segment-classification.md
|
||||
│ └── user/
|
||||
│ └── local/
|
||||
├── raw-sources/
|
||||
│ └── warehouse/
|
||||
│ └── <syncId>/ # Database ingest artifacts and reports
|
||||
└── .ktx/
|
||||
└── db.sqlite # Local state (git-ignored)
|
||||
├── ktx.yaml # Project configuration
|
||||
├── semantic-layer/<connection-id>/ # YAML semantic sources
|
||||
├── wiki/global/ # Shared business context
|
||||
├── wiki/user/<user-id>/ # User-scoped notes
|
||||
├── raw-sources/<connection-id>/ # Ingest artifacts and reports
|
||||
└── .ktx/ # Local state and secrets, git-ignored
|
||||
```
|
||||
|
||||
Semantic sources and wiki pages are committed to git. The `.ktx/` directory
|
||||
holds ephemeral state and is git-ignored - delete it and KTX rebuilds on the
|
||||
next run.
|
||||
Commit `ktx.yaml`, `semantic-layer/`, and `wiki/`. Keep `.ktx/` local.
|
||||
|
||||
### Build demo warehouse context
|
||||
## Agent Usage
|
||||
|
||||
Database ingest artifacts are written under `raw-sources/warehouse/<syncId>/`
|
||||
in the project directory.
|
||||
Setup can install KTX instructions for Claude Code, Codex, Cursor, OpenCode,
|
||||
and universal `.agents` clients:
|
||||
|
||||
```bash
|
||||
ktx ingest warehouse --project-dir "$PROJECT_DIR" --fast
|
||||
ktx status --project-dir "$PROJECT_DIR"
|
||||
ktx setup --agents --target codex
|
||||
```
|
||||
|
||||
For non-SQLite drivers, prefer credential references such as `--url env:NAME`
|
||||
or `--url file:PATH` over literal credential URLs.
|
||||
|
||||
## Managed Python runtime
|
||||
|
||||
KTX installs its Python runtime only when a Python-backed command needs it.
|
||||
The runtime lives outside the npm cache, is versioned by the installed CLI
|
||||
version, and is managed by `ktx dev runtime` commands.
|
||||
|
||||
KTX requires `uv` on `PATH` to create the managed runtime. Install `uv` with
|
||||
your system package manager or the official installer before running Python-
|
||||
backed KTX commands. KTX doesn't download `uv` automatically; run
|
||||
`ktx dev runtime status` if runtime installation fails:
|
||||
Agent-facing workflows typically start with:
|
||||
|
||||
```bash
|
||||
ktx dev runtime install --yes
|
||||
ktx dev runtime status
|
||||
ktx dev runtime start
|
||||
ktx dev runtime stop
|
||||
ktx sl search "revenue" --json
|
||||
ktx wiki search "refund policy" --json
|
||||
ktx sl query --connection-id warehouse --measure orders.revenue --format sql
|
||||
```
|
||||
|
||||
The release artifact manifest contains the public npm tarball and the bundled `kaelio-ktx`
|
||||
runtime wheel. The `python/ktx-sl` and `python/ktx-daemon` directories remain
|
||||
source packages for development, not public release artifacts.
|
||||
|
||||
## Use KTX with agents
|
||||
|
||||
KTX integrates with coding agents through CLI skills. The setup wizard
|
||||
configures this automatically.
|
||||
|
||||
**CLI skills** - the agent calls `ktx` commands directly through a skill file
|
||||
installed in your agent's config (e.g., `.claude/skills/ktx/SKILL.md`):
|
||||
|
||||
```bash
|
||||
ktx sl query --measure orders.revenue --dimension orders.status --format sql
|
||||
ktx wiki search "revenue definition"
|
||||
ktx sl validate orders
|
||||
```
|
||||
|
||||
Supported agents: Claude Code, Codex, Cursor, OpenCode, and any agent that
|
||||
reads `.agents/` skills.
|
||||
|
||||
## Workspace packages
|
||||
|
||||
| Package | Purpose |
|
||||
|---------|---------|
|
||||
| `packages/cli` | CLI entry point |
|
||||
| `packages/context` | Core context engine |
|
||||
| `packages/llm` | LLM and embedding providers |
|
||||
| `packages/connector-bigquery` | BigQuery scan connector |
|
||||
| `packages/connector-clickhouse` | ClickHouse scan connector |
|
||||
| `packages/connector-mysql` | MySQL scan connector |
|
||||
| `packages/connector-postgres` | Postgres scan connector |
|
||||
| `packages/connector-snowflake` | Snowflake scan connector |
|
||||
| `packages/connector-sqlite` | SQLite scan connector |
|
||||
| `packages/connector-sqlserver` | SQL Server scan connector |
|
||||
| `python/ktx-sl` | Semantic-layer query planning |
|
||||
| `python/ktx-daemon` | Portable compute service |
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
|
|
@ -204,7 +103,7 @@ pnpm run build
|
|||
pnpm run check
|
||||
```
|
||||
|
||||
Use the development CLI for local testing:
|
||||
Use the development CLI locally:
|
||||
|
||||
```bash
|
||||
pnpm run setup:dev
|
||||
|
|
@ -212,30 +111,28 @@ pnpm run link:dev
|
|||
ktx-dev --help
|
||||
```
|
||||
|
||||
### Debug LLM traces
|
||||
KTX is a pnpm + uv workspace:
|
||||
|
||||
KTX can capture local AI SDK DevTools traces for LLM calls that run through the
|
||||
KTX provider. Enable it with an environment flag when running an LLM-backed
|
||||
command:
|
||||
- TypeScript packages live in `packages/*`
|
||||
- CLI source lives in `packages/cli`
|
||||
- Python runtime source lives in `python/ktx-sl` and `python/ktx-daemon`
|
||||
- Public docs live in `docs-site/content/docs`
|
||||
|
||||
Useful checks:
|
||||
|
||||
```bash
|
||||
KTX_AI_DEVTOOLS_ENABLED=true ktx ingest warehouse --project-dir "$PROJECT_DIR" --deep
|
||||
pnpm run type-check
|
||||
pnpm run test
|
||||
pnpm run dead-code
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Traces are written to `.devtools/generations.json` under the current working
|
||||
directory. To inspect them, run:
|
||||
## Docs
|
||||
|
||||
```bash
|
||||
pnpm dlx @ai-sdk/devtools
|
||||
```
|
||||
|
||||
Then open `http://localhost:4983`. These traces are local-development-only and
|
||||
store prompts, model outputs, tool arguments/results, and raw provider payloads
|
||||
in plain text. Do not enable this in production or for sensitive runs.
|
||||
|
||||
The repository uses `pnpm` for TypeScript packages and `uv` for Python
|
||||
packages. See [Contributing](docs-site/content/docs/community/contributing.mdx)
|
||||
for full development setup, testing, and PR guidelines.
|
||||
- [Quickstart](docs-site/content/docs/getting-started/quickstart.mdx)
|
||||
- [CLI Reference](docs-site/content/docs/cli-reference/index.mdx)
|
||||
- [Building Context](docs-site/content/docs/guides/building-context.mdx)
|
||||
- [Contributing](docs-site/content/docs/community/contributing.mdx)
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue