apunkt/ktx
1
0
Fork 0
mirror of https://github.com/Kaelio/ktx.git synced 2026-06-07 07:55:13 +02:00
Code Issues Projects Releases Packages Wiki Activity

docs: add ktx skills.sh setup skill (#227)

Browse source
This commit is contained in:
Andrey Avtomonov 2026-05-28 12:28:10 +02:00 • committed by GitHub
parent 27842e14a9
commit 39f94f39ff
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 177 additions and 246 deletions
Download patch file Download diff file Expand all files Collapse all files

5
README.md
View file

@ -119,9 +119,8 @@ Agent integration ready: yes (codex:project)
> your project directory: > your project directory:
> >
> ```text > ```text
> Follow instructions from > Run npx skills add Kaelio/ktx --skill ktx and use the ktx skill to install
> https://docs.kaelio.com/ktx/docs/agents-setup.md > and configure ktx in this project.
> to install and configure ktx
> ``` > ```
> [!IMPORTANT] > [!IMPORTANT]

18
docs-site/app/llms.mdx/docs/[[...slug]]/route.ts
View file

@ -3,11 +3,6 @@ import {
getLlmDocsPages, getLlmDocsPages,
getPageMarkdown, getPageMarkdown,
} from "@/lib/llm-docs"; } from "@/lib/llm-docs";
import {
agentSetupSlug,
isAgentSetupSlug,
readAgentSetupMarkdown,
} from "@/lib/agent-setup-markdown";
export const dynamic = "force-static"; export const dynamic = "force-static";
@ -16,14 +11,6 @@ export async function GET(
props: { params: Promise<{ slug?: string[] }> }, props: { params: Promise<{ slug?: string[] }> },
) { ) {
const params = await props.params; const params = await props.params;
if (isAgentSetupSlug(params.slug)) {
return new Response(await readAgentSetupMarkdown(), {
headers: {
"Content-Type": "text/markdown; charset=utf-8",
},
});
}
const page = getLlmDocsPage(params.slug); const page = getLlmDocsPage(params.slug);
if (!page) { if (!page) {
return new Response("Documentation page not found.\n", { return new Response("Documentation page not found.\n", {
@ -42,8 +29,5 @@ export async function GET(
} }
export function generateStaticParams() { export function generateStaticParams() {
return [ return getLlmDocsPages().map((page) => ({ slug: page.slug }));
...getLlmDocsPages().map((page) => ({ slug: page.slug })),
{ slug: [...agentSetupSlug] },
];
} }

201
docs-site/content/agents-setup.md
View file

@ -1,201 +0,0 @@
# Goal
Set up **ktx** from scratch end-to-end as a fully autonomous, agent-driven replacement for the interactive `ktx setup` wizard. Detect the environment, install missing prerequisites, ask the user only for information you genuinely need (which connections to add, credentials), write a valid configuration, verify it works, and run a fast ingest. Keep the user updated throughout.
# Operating principles
- **Be autonomous.** Detect, decide, and act. Only ask the user when you need information that only they can provide: project location, which databases/sources to connect, credentials, and similar choices.
- **Stream short status updates.** Before each major phase ("Checking prerequisites…", "Installing uv…", "Configuring warehouse connection…", "Running fast ingest…") print a one-line update. Not chatty - just enough that the user can see what's happening.
- **Verify against docs, never guess.** CLI flags, config keys, and command names must come from the docs or from `ktx <command> --help`. If something looks wrong or missing, say so explicitly.
- **Print every command you run and its exit code.** Terse, not silent.
- **Fail loudly with cause + fix.** When a command fails: capture the exact error, identify the cause, change something, retry. Never retry an unchanged command. Exceptions for *known soft-failures* are listed in Phase 4 - handle those without retrying.
- **No LLM-based ingestion in this flow.** Only `--fast` ingest. The user can run `--deep` later.
- **Platform-agnostic.** Detect the host OS first and pick the right install commands / path syntax. Anything path- or shell-specific must branch on OS.
# Authoritative docs
**ktx** docs are served at `https://docs.kaelio.com/ktx/`. **Start by fetching `https://docs.kaelio.com/ktx/llms.txt`** to discover the docs map. Scan it for a "troubleshooting" entry - if one exists, read it **before** running install/setup so you can apply known fixes preemptively rather than after failing. If no troubleshooting page is listed (current state of the docs), proceed. Then fetch any other `.md` pages you need (setup, ingest, status, connection types). **Never invent CLI flags or config keys** - verify against the docs or `ktx --help` / `ktx <subcommand> --help`.
> **Note on the `ktx status` JSON example in the docs.** The docs page for `ktx status` shows an example shaped like `{"title": "...", "checks": [...]}`. That example is outdated. The real CLI output uses a top-level `verdict` field plus a `connections[]` array - see Phase 5 for the canonical success criteria. Trust the shape in this prompt over the docs example.
# Workflow
## Phase 1 - Detect environment
Determine the host OS (e.g. via `uname -s`, `process.platform`, or `$env:OS`). Use the right install commands per OS for the rest of this flow.
| Tool | macOS / Linux | Windows (PowerShell) |
|------|---------------|----------------------|
| `uv` | `curl -LsSf https://astral.sh/uv/install.sh \| sh` then re-source shell env | `irm https://astral.sh/uv/install.ps1 \| iex` |
| Node.js | use system / fnm / nvm - **do not** auto-install | use system / nvm-windows - **do not** auto-install |
| **ktx** CLI | `npm install -g …` (see Phase 2) | `npm install -g …` (see Phase 2) |
If Node.js is missing, **stop and ask the user** to install it (https://nodejs.org/). Do not attempt to auto-install Node.
## Phase 2 - Verify and install prerequisites
Check each tool in order; install only if missing.
1. **Node.js** - run `node --version`. Require >= 22. If missing or older, stop and instruct the user.
2. **`uv`** - run `uv --version`. If missing, run the OS-appropriate install command, then re-source the shell environment (`export PATH="$HOME/.local/bin:$PATH"` on Linux/macOS) so `uv` is on `PATH`.
3. **ktx CLI** -
- Install ktx with `npm install -g @kaelio/ktx`
- Verify with `ktx --version`.
Print one status line per tool ("✓ uv 0.11.15 found", "Installing uv…", "✓ ktx 0.x.y installed").
## Phase 3 - Gather user choices
Ask the user (grouped if your harness supports it; otherwise sequentially):
1. **Project directory.** Default: current working directory. Confirm before continuing.
2. **LLM provider.** Default: `claude-code` with model `sonnet` (the user is already inside Claude Code; no extra API key needed). Offer `anthropic` (paste API key, stored as `env:` or `file:` ref) and `vertex` (GCP project + location) as alternatives. Skip if defaults are accepted.
3. **Embeddings backend.** Default: `sentence-transformers` (local, no API key, managed Python runtime). Offer `openai` only if the user has a key.
4. **Database connections.** Ask how many to add, then loop. For each, collect:
- Connection name (e.g. `warehouse`, `analytics`).
- Driver: one of `sqlite`, `postgres`, `mysql`, `sqlserver`, `bigquery`, `snowflake`.
- Connection URL/DSN (or service-account file for BigQuery). Accept `env:VAR_NAME` or `file:/abs/path` to avoid pasting raw secrets.
- **Heads-up for the user**: even if they paste a literal URL, **ktx** will silently relocate it into `<project>/.ktx/secrets/<connection>-url` and rewrite `ktx.yaml` to `url: file:…` - this is correct, secure behavior and not a bug.
- Schemas / datasets to include (postgres / sqlserver / snowflake / bigquery only).
- Optional `enabled_tables` allowlist if the user wants to scope ingest to specific tables.
5. **Context sources** (dbt, Metabase, Looker, LookML, MetricFlow, Notion). Default: none. Ask only if the user mentions them.
## Phase 4 - Configure the project
Drive the existing wizard non-interactively (verify exact flag names with `ktx setup --help` and the docs - the automation flags are hidden from help but accepted):
```
ktx setup \
--project-dir <path> \
--no-input --yes \
--llm-backend <claude-code|anthropic|vertex> --llm-model <model> \
[--anthropic-api-key-env ANTHROPIC_API_KEY | --anthropic-api-key-file <path>] \
[--vertex-project <p> --vertex-location <loc>] \
--embedding-backend <sentence-transformers|openai> \
[--embedding-api-key-env OPENAI_API_KEY] \
--skip-sources \
--database <driver> --database-connection-id <name> --database-url <url|env:VAR|file:/path> \
[--database-schema <schema> …]
```
Notes on the flags above:
- **Project creation is automatic with `--no-input --yes`.** When
`ktx.yaml` exists, setup resumes it. When it doesn't exist, setup creates it
at `--project-dir`.
- **`--database-connection-id` is dual-purpose.** With `--database` or
`--database-url`, it names the new connection. Without those flags, it
selects an existing connection id.
- **Configure one new database connection per setup command.** If the user
wants multiple new connections, run setup again for each connection.
- **You don't need `--skip-agents` in this flow.** The agent integration step
is opt-in: setup leaves it alone unless you pass `--agents --target
<target>`.
- **`--skip-sources`** is correct and is the documented way to leave context sources unconfigured.
### Known soft-failure: `ktx setup` exits 1 after a successful fast build
When you select a configuration that only does fast ingest, `ktx setup`'s final readiness verification fails with:
```
ktx context build did not pass agent-readiness verification.
<connection>: deep database context has not completed.
```
This is **expected** and **does not mean setup failed**. Treat the exit code as a soft-failure **only if all of the following hold**:
- The build log shows the fast ingest reached `[100%] Scan completed` for every configured connection.
- `ktx connection test <name>` (run next) exits 0 for every connection.
- `ktx status --json --no-input` reports `verdict: "ready"`.
If those three conditions hold, proceed to Phase 5 without retrying setup, and **do not** switch to `--deep` to "fix" the readiness gate - deep ingest is explicitly out of scope. Mention this in the final report under "Docs / CLI gaps" so the user is aware.
If any of those three conditions do not hold, this is a real failure - capture the error, fetch the relevant docs page, fix the cause, retry.
After `ktx setup` writes `ktx.yaml`, edit it directly for anything flags don't cover:
- Per-connection `enabled_tables` allowlist (snake_case, under `connections.<name>.enabled_tables`).
- Any advanced settings the user requested.
Use a YAML-aware editor (e.g. `uv run python -c "import yaml; …"`) - do not hand-edit blindly.
## Phase 5 - Verify
`ktx setup` already runs a fast ingest of every database connection it configures, so you do not need to re-ingest by default. For each configured connection:
```
ktx connection test <connection-name> # must exit 0
```
Only re-run ingest if setup's build log did **not** reach 100% for that connection:
```
ktx ingest <connection-name> --fast --no-input
```
**Mutex warning on `ktx ingest`**: passing both `--yes` and `--no-input` fails with `Choose only one runtime install mode: --yes or --no-input`. Setup already installed the managed Python runtime, so pass **only `--no-input`** to `ktx ingest`. (`--yes` is only needed when an ingest invocation has to install the runtime itself, which is not the case here.)
Then run the global health check:
```
ktx status --json --no-input
```
Success requires (canonical shape - supersedes the example in the docs):
- `verdict: "ready"` at the top of the JSON.
- Every `connections[].status === "ok"`.
- `ktx connection test <name>` exited 0 for every connection.
Do **not** run `--deep` ingest in this flow - that requires LLM time and is out of scope.
### Optional: directly probe the ktx daemon
If the user asks for stronger verification that `sentence-transformers` is actually serving (not just that setup said "ok"), do all of:
1. `ktx admin runtime status --json` → expect `"kind": "ready"` and `"features": [..., "local-embeddings"]`.
2. `pgrep -fa ktx-daemon` → expect a process running `ktx-daemon serve-http`.
3. `curl -sS http://127.0.0.1:<port>/health` → expect HTTP 200 with `{"status":"healthy",…}`.
4. `curl -sS -X POST http://127.0.0.1:<port>/embeddings/compute -H 'content-type: application/json' -d '{"text":"hello"}'` → expect `{"embedding": [...384 floats...]}`.
Discover the port from setup's log line `Started ktx daemon: http://127.0.0.1:<port>` or from the daemon's OpenAPI at `GET /openapi.json`. Note: the routes are `/health` and `/embeddings/compute` - not `/healthz` or `/embeddings`.
## Phase 6 - Final report
Print a structured report:
```
ktx SETUP COMPLETE
Project: <path>
LLM: <backend> / <model>
Embeddings: <backend> / <model>
Runtime: managed Python ✓ (if the ktx daemon was started)
Connections:
- <name> (<driver>) status=ok schemas=[…] tables=<N>
- …
Sources: <list or "none">
Verdict: ready
```
Then **Next steps** (copy-pasteable):
1. Enrich with AI descriptions and embeddings: `ktx ingest <connection> --deep` (several minutes per connection).
2. Add more connections later by rerunning this setup or via `ktx setup --database … --database-connection-id …`.
3. Configure context sources (dbt, Metabase, Looker, LookML, MetricFlow, Notion) - see `ktx setup --help` for `--source …` flags.
4. Install agent integration: `ktx setup --agents --target <claude-code|claude-desktop|codex|cursor|opencode|universal>` (with optional `--global` for `claude-code`/`codex`).
5. Connect the agent / MCP: see docs at `https://docs.kaelio.com/ktx/`.
Under **Docs / CLI gaps to flag** include any of these that applied during your run:
- `ktx setup` exits non-zero after a successful fast build (deep-readiness gate); status reports ready.
- `ktx ingest` rejects `--yes` and `--no-input` together; docs don't note the conflict.
- `ktx status --json` real shape (`verdict`, `connections[]`) doesn't match the example in the docs page.
- The pasted DB URL was moved to `.ktx/secrets/<name>-url` automatically.
End with a single line: `RESULT: PASS` or `RESULT: FAIL - <one-line reason>`.
# Operating rules (recap)
- Print every command you run and its exit code. Status updates may be terse, but never silent.
- On failure: capture the error, fetch the relevant docs page, fix the cause, retry. Never retry an unchanged command.
- Known soft-failures (listed in Phase 4 and Phase 5) are not real failures - handle them as documented; do not retry or escalate.
- If you find a docs/CLI gap ("docs say X but CLI does Y"), call it out in the final report.
- Never commit credentials - **ktx** accepts `env:` and `file:` references; prefer those. **ktx** will also auto-relocate literal URLs into `.ktx/secrets/`, but that does not protect anyone who pasted the URL into chat history.

3
docs-site/content/docs/ai-resources/prompt-recipes.mdx
View file

@ -14,7 +14,8 @@ Read https://docs.kaelio.com/ktx/llms.txt first. Then fetch only the ktx Markdow
## Set up a project ## Set up a project
```text ```text
Set up ktx in this repository. Start by reading /docs/ai-resources/agent-quickstart.md and /docs/getting-started/quickstart.md. Install the published CLI with npm; use pnpm only when working from a ktx source checkout. After setup, run ktx status and summarize which steps are complete, which files changed, and what still needs credentials or user input. Run npx skills add Kaelio/ktx --skill ktx and use the ktx skill to install
and configure ktx in this project.
``` ```
## Find a command ## Find a command

21
docs-site/content/docs/getting-started/quickstart.mdx
View file

@ -103,11 +103,8 @@ If you're a coding assistant choosing a docs route, start with the
</div> </div>
<div className="mt-2 text-sm leading-6 text-fd-muted-foreground"> <div className="mt-2 text-sm leading-6 text-fd-muted-foreground">
You can ask an agent such as Claude Code, Codex, Cursor, or OpenCode to You can ask an agent such as Claude Code, Codex, Cursor, or OpenCode to
install and configure **ktx** for you. The{' '} install and configure **ktx** for you. The installable **ktx** skill tells
<a href="/ktx/docs/agents-setup.md" className="font-medium underline"> the agent how to check prerequisites, ask only for credentials or
agent setup Markdown prompt
</a>{' '}
tells the agent how to check prerequisites, ask only for credentials or
connection choices, run <code>ktx setup</code>, verify connections, and connection choices, run <code>ktx setup</code>, verify connections, and
report the result. report the result.
</div> </div>
@ -120,16 +117,18 @@ If you're a coding assistant choosing a docs route, start with the
Prompt Prompt
</span> </span>
<CopyButton <CopyButton
text={`Follow instructions from text={[
https://docs.kaelio.com/ktx/docs/agents-setup.md 'Run npx skills add Kaelio/ktx --skill ktx and use the ktx skill',
to install and configure ktx`} 'to install and configure ktx in this project.',
].join(' ')}
className="-my-1" className="-my-1"
/> />
</div> </div>
<div className="p-3 font-mono text-sm leading-6 text-fd-foreground"> <div className="p-3 font-mono text-sm leading-6 text-fd-foreground">
<div>Follow instructions from</div> <div>
<div className="break-all">https://docs.kaelio.com/ktx/docs/agents-setup.md</div> Run npx skills add Kaelio/ktx --skill ktx and use the ktx skill to
<div>to install and configure ktx</div> install and configure ktx in this project.
</div>
</div> </div>
</div> </div>
</div> </div>

12
docs-site/lib/agent-setup-markdown.ts
View file

@ -1,12 +0,0 @@
import { readFile } from "node:fs/promises";
import { join } from "node:path";
export const agentSetupSlug = ["agents-setup"] as const;
export function isAgentSetupSlug(slug: string[] | undefined) {
return slug?.length === 1 && slug[0] === agentSetupSlug[0];
}
export function readAgentSetupMarkdown() {
return readFile(join(process.cwd(), "content/agents-setup.md"), "utf8");
}

3
docs-site/lib/llm-docs.ts
View file

@ -52,8 +52,9 @@ ktx provides semantic-layer files, warehouse scans, wiki pages, provenance, and
## Agent Entry Points ## Agent Entry Points
- Installable setup skill: run \`npx skills add Kaelio/ktx --skill ktx\` from
the project you want to configure.
${link("/docs/ai-resources/agent-quickstart", "Agent Quickstart", "Task-first route for coding assistants using ktx")} ${link("/docs/ai-resources/agent-quickstart", "Agent Quickstart", "Task-first route for coding assistants using ktx")}
${link("/docs/agents-setup", "Agent Setup", "Copy-pasteable prompt for agents installing and configuring ktx")}
${link("/docs/ai-resources/markdown-access", "Markdown Access", "Fetch ktx docs as llms.txt, llms-full.txt, or per-page Markdown")} ${link("/docs/ai-resources/markdown-access", "Markdown Access", "Fetch ktx docs as llms.txt, llms-full.txt, or per-page Markdown")}
${link("/docs/ai-resources/agent-instructions", "Agent Instructions", "Suggested instructions for coding assistants that need to read and cite ktx docs")} ${link("/docs/ai-resources/agent-instructions", "Agent Instructions", "Suggested instructions for coding assistants that need to read and cite ktx docs")}

11
skills.sh.json Normal file
View file

@ -0,0 +1,11 @@
{
"$schema": "https://skills.sh/schemas/skills.sh.schema.json",
"notGrouped": "bottom",
"groupings": [
{
"title": "ktx",
"description": "Skills for installing, configuring, and operating ktx.",
"skills": ["ktx"]
}
]
}

142
skills/ktx/SKILL.md Normal file
View file

@ -0,0 +1,142 @@
---
name: ktx
description: Use when installing, configuring, verifying, or debugging ktx in a project, including ktx setup, ktx.yaml, database connectors, embeddings, agent integration, ingest, and ktx status checks.
---
# ktx
Install and configure **ktx**, the open-source context layer for data agents.
Use this skill when a user wants an agent to add **ktx** to a project, connect
data sources, build initial context, install agent rules, or troubleshoot a
local **ktx** setup.
## Operating rules
- Act autonomously when the user asks you to install or configure **ktx**.
- Ask only for choices or values you cannot infer: project directory,
connection targets, credentials, account identifiers, and source selections.
- Never ask the user to paste secrets when an `env:VAR_NAME` or `file:/path`
reference would work.
- Do not commit `.ktx/secrets/*` or pasted credentials.
- Verify CLI flags and config keys with `ktx --help`, `ktx <command> --help`,
or the docs at `https://docs.kaelio.com/ktx/` before using unfamiliar
options.
- Print or report each command you run and its result when doing setup work.
- If a command fails, identify the cause and change something before retrying.
## Install workflow
Use this workflow for a new or resumed project setup:
1. Confirm the project directory. Default to the current working directory.
2. Check prerequisites:
- Node.js with `node --version`; require Node 22 or newer.
- `uv` with `uv --version`; install it only if missing and local Python
runtime features are needed.
- **ktx** with `ktx --version`; install the published CLI if missing.
3. Install the published CLI when needed:
```bash
npm install -g @kaelio/ktx
```
4. Run interactive setup when the user is present:
```bash
ktx setup
```
5. For scripted setup, prefer `ktx setup --no-input --yes` with explicit flags.
Verify exact flags with `ktx setup --help` and the docs first.
6. Configure one new database connection per scripted setup command. For
multiple connections, rerun setup once per connection.
7. Run fast ingest by default. Do not run deep ingest unless the user asks for
LLM-backed enrichment.
8. Install or repair agent integration after project setup:
```bash
ktx setup --agents
```
9. Verify readiness:
```bash
ktx status
```
Use `ktx status --json` when you need structured success criteria.
## Common setup choices
Default choices are usually:
- LLM: `claude-code` if the user is already running Claude Code, otherwise ask.
- Embeddings: `sentence-transformers` for local embeddings with no API key, or
`openai` when the user wants hosted embeddings and has an API key.
- Databases: SQLite, PostgreSQL, MySQL, SQL Server, BigQuery, Snowflake, or
ClickHouse.
- Context sources: dbt, MetricFlow, LookML, Looker, Metabase, or Notion.
Use `env:` or `file:` references for credentials:
```bash
ktx setup \
--project-dir ./analytics \
--no-input \
--yes \
--database postgres \
--database-connection-id warehouse \
--database-url env:DATABASE_URL \
--database-schema public
```
Then build or refresh fast context if setup did not already do it:
```bash
ktx ingest warehouse --fast --no-input
```
## Files to inspect
- `ktx.yaml`: project configuration.
- `.ktx/secrets/*`: local secret files. Never commit them.
- `semantic-layer/<connection-id>/*.yaml`: semantic sources for SQL
compilation.
- `wiki/**/*.md`: project context pages for agents.
- `.claude/skills/ktx/`, `.agents/skills/ktx/`, `.cursor/rules/ktx.mdc`, and
`.opencode/commands/ktx.md`: generated agent integration files.
## Verification
After setup, run the smallest checks that cover the configured surface:
```bash
ktx connection test <connection-id>
ktx status --json
```
Success means the project is ready, configured connections report healthy, and
the agent integration target requested by the user is installed. If fast setup
completed but deep context readiness is still missing, report that as the next
optional enrichment step rather than retrying setup unchanged.
## Final report
End setup work with a concise report:
```text
ktx SETUP COMPLETE
Project: <path>
LLM: <backend> / <model>
Embeddings: <backend> / <model>
Connections: <name> (<driver>) status=<ok|warn|fail>
Sources: <list or none>
Verdict: <ready|needs action>
Next:
1. <copy-pasteable command or action>
2. <copy-pasteable command or action>
RESULT: PASS
```

7
skills/ktx/agents/openai.yaml Normal file
View file

@ -0,0 +1,7 @@
interface:
display_name: "ktx"
short_description: "Install and configure ktx for data agents"
default_prompt: "Use $ktx to install and configure ktx in this project."
policy:
allow_implicit_invocation: true