mirror of
https://github.com/Kaelio/ktx.git
synced 2026-06-19 08:28:06 +02:00
feat(cli): clean up wiki and sl commands
This commit is contained in:
Andrey Avtomonov
parent
e1e9c4af91
commit
67b587f5d0
18 changed files with 311 additions and 585 deletions
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: "ktx sl"
|
||||
description: "List, read, validate, query, or write semantic-layer sources."
|
||||
description: "List, search, validate, or query semantic-layer sources."
|
||||
---
|
||||
|
||||
Interact with your project's semantic layer. Semantic sources are YAML definitions that describe your tables, columns, measures, joins, and grain — the vocabulary agents use to generate correct SQL.
|
||||
|
|
@ -16,9 +16,8 @@ ktx sl <subcommand> [options]
|
|||
| Subcommand | Description |
|
||||
|-----------|-------------|
|
||||
| `list` | List semantic-layer sources |
|
||||
| `read <sourceName>` | Read a semantic-layer source |
|
||||
| `search <query>` | Search semantic-layer sources |
|
||||
| `validate <sourceName>` | Validate a semantic-layer source against the database schema |
|
||||
| `write <sourceName>` | Write a semantic-layer source |
|
||||
| `query` | Compile or execute a semantic-layer query |
|
||||
|
||||
## Options
|
||||
|
|
@ -28,16 +27,17 @@ ktx sl <subcommand> [options]
|
|||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--connection-id <id>` | Filter by KTX connection id | — |
|
||||
| `--query <text>` | Search source names and descriptions | — |
|
||||
| `--output <mode>` | Output mode: `pretty` (default in TTY), `plain` (TSV), or `json` | `pretty` |
|
||||
| `--json` | Shortcut for `--output=json` (overrides `--output`) | `false` |
|
||||
|
||||
### `sl read`
|
||||
### `sl search`
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--connection-id <id>` | KTX connection id (required) | — |
|
||||
| `--json` | Print JSON output | `false` |
|
||||
| `--connection-id <id>` | Filter by KTX connection id | — |
|
||||
| `--limit <number>` | Maximum search results | — |
|
||||
| `--output <mode>` | Output mode: `pretty` (default in TTY), `plain` (TSV), or `json` | `pretty` |
|
||||
| `--json` | Shortcut for `--output=json` (overrides `--output`) | `false` |
|
||||
|
||||
### `sl validate`
|
||||
|
||||
|
|
@ -45,13 +45,6 @@ ktx sl <subcommand> [options]
|
|||
|------|-------------|---------|
|
||||
| `--connection-id <id>` | KTX connection id (required) | — |
|
||||
|
||||
### `sl write`
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--connection-id <id>` | KTX connection id (required) | — |
|
||||
| `--yaml <yaml>` | Semantic-layer source YAML content (required) | — |
|
||||
|
||||
### `sl query`
|
||||
|
||||
| Flag | Description | Default |
|
||||
|
|
@ -82,20 +75,11 @@ ktx sl list --connection-id my-warehouse
|
|||
ktx sl list --json
|
||||
|
||||
# Search sources as JSON
|
||||
ktx sl list --json --query "revenue"
|
||||
|
||||
# Read a source definition
|
||||
ktx sl read orders --connection-id my-warehouse
|
||||
|
||||
# Read a source definition as JSON
|
||||
ktx sl read orders --connection-id my-warehouse --json
|
||||
ktx sl search "revenue" --json
|
||||
|
||||
# Validate a source against the live schema
|
||||
ktx sl validate orders --connection-id my-warehouse
|
||||
|
||||
# Write a new source from YAML
|
||||
ktx sl write customers --connection-id my-warehouse --yaml "$(cat sources/customers.yaml)"
|
||||
|
||||
# Compile a query and view the generated SQL
|
||||
ktx sl query \
|
||||
--connection-id my-warehouse \
|
||||
|
|
@ -159,5 +143,5 @@ Semantic-layer commands return human-readable output by default. Use `--json` or
|
|||
|-------|-------|----------|
|
||||
| Source not found | Source name or connection id is wrong | Run `ktx sl list --json` and retry with an exact source name and connection id |
|
||||
| Validation fails | YAML references missing columns, invalid joins, or invalid SQL expressions | Fix the source YAML and rerun `ktx sl validate` |
|
||||
| Query compile fails | Measure, dimension, filter, or segment name is invalid | Read the source with `ktx sl read`, then retry using declared fields |
|
||||
| Query compile fails | Measure, dimension, filter, or segment name is invalid | Search sources with `ktx sl search`, inspect the source YAML in your project files, then retry using declared fields |
|
||||
| Execution returns too many rows | `--max-rows` is missing or too high | Add `--max-rows` with a bounded value before executing |
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: "ktx wiki"
|
||||
description: "List, read, search, or write knowledge pages."
|
||||
description: "List or search knowledge pages."
|
||||
---
|
||||
|
||||
Manage knowledge pages in your KTX project. Knowledge pages are Markdown documents that capture business definitions, rules, and gotchas. Agents search them for context when answering questions about your data.
|
||||
|
|
@ -16,9 +16,7 @@ ktx wiki <subcommand> [options]
|
|||
| Subcommand | Description |
|
||||
|-----------|-------------|
|
||||
| `list` | List local wiki pages |
|
||||
| `read <key>` | Read one local wiki page |
|
||||
| `search <query>` | Search local wiki pages |
|
||||
| `write <key>` | Write one local wiki page |
|
||||
|
||||
## Options
|
||||
|
||||
|
|
@ -29,13 +27,6 @@ ktx wiki <subcommand> [options]
|
|||
| `--json` | Print JSON output | `false` |
|
||||
| `--user-id <id>` | Local user id | `local` |
|
||||
|
||||
### `wiki read`
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--json` | Print JSON output | `false` |
|
||||
| `--user-id <id>` | Local user id | `local` |
|
||||
|
||||
### `wiki search`
|
||||
|
||||
| Flag | Description | Default |
|
||||
|
|
@ -44,18 +35,6 @@ ktx wiki <subcommand> [options]
|
|||
| `--user-id <id>` | Local user id | `local` |
|
||||
| `--limit <number>` | Maximum search results | — |
|
||||
|
||||
### `wiki write`
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--user-id <id>` | Local user id | `local` |
|
||||
| `--scope <scope>` | Scope: `global` or `user` | `global` |
|
||||
| `--summary <summary>` | Wiki page summary (required) | — |
|
||||
| `--content <content>` | Wiki page content (required) | — |
|
||||
| `--tag <tag>` | Wiki tag; repeatable | — |
|
||||
| `--ref <ref>` | Wiki ref; repeatable | — |
|
||||
| `--sl-ref <ref>` | Semantic-layer ref; repeatable | — |
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
|
|
@ -65,48 +44,16 @@ ktx wiki list
|
|||
# List all wiki pages as JSON
|
||||
ktx wiki list --json
|
||||
|
||||
# Read a specific wiki page
|
||||
ktx wiki read revenue-definitions
|
||||
|
||||
# Read a specific wiki page as JSON
|
||||
ktx wiki read revenue-definitions --json
|
||||
|
||||
# Search wiki pages
|
||||
ktx wiki search "monthly recurring revenue"
|
||||
|
||||
# Search wiki pages as JSON
|
||||
ktx wiki search "monthly recurring revenue" --json --limit 10
|
||||
|
||||
# Write a global knowledge page
|
||||
ktx wiki write revenue-definitions \
|
||||
--summary "Canonical revenue metric definitions" \
|
||||
--content "## MRR\nMonthly Recurring Revenue is calculated as..."
|
||||
|
||||
# Write a user-scoped knowledge page
|
||||
ktx wiki write my-notes \
|
||||
--scope user \
|
||||
--summary "Personal analysis notes" \
|
||||
--content "Things to check when revenue numbers look off..."
|
||||
|
||||
# Write a page with tags and references
|
||||
ktx wiki write churn-rules \
|
||||
--summary "Churn calculation business rules" \
|
||||
--content "A customer is considered churned when..." \
|
||||
--tag finance \
|
||||
--tag retention \
|
||||
--sl-ref customers \
|
||||
--sl-ref subscriptions
|
||||
|
||||
# Write a page with external references
|
||||
ktx wiki write data-freshness \
|
||||
--summary "Data pipeline SLAs and freshness guarantees" \
|
||||
--content "The orders table refreshes every 15 minutes..." \
|
||||
--ref "https://wiki.example.com/data-pipelines"
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
Wiki commands print local knowledge pages and search results. Agents should search first, then read the most relevant page by key.
|
||||
Wiki commands print local knowledge pages and search results.
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -127,7 +74,5 @@ Wiki commands print local knowledge pages and search results. Agents should sear
|
|||
|
||||
| Error | Cause | Recovery |
|
||||
|-------|-------|----------|
|
||||
| Search returns no results | The query terms do not match summaries, tags, or content | Retry with business synonyms, then create a page if the knowledge is missing |
|
||||
| Read fails for a key | The page key is wrong or scoped to a different user | Run `ktx wiki list` or search again to get the exact key |
|
||||
| Write fails due to missing fields | `--summary` or `--content` was omitted | Pass both fields, and keep the summary short enough for search results |
|
||||
| Agent writes duplicate pages | It did not search existing pages first | Always run `ktx wiki search` before `ktx wiki write` |
|
||||
| Search returns no results | The query terms do not match summaries, tags, or content | Retry with business synonyms or run ingest to capture more context |
|
||||
| A page is missing | The page has not been created by ingest or memory capture yet | Run ingest, then search again with `ktx wiki search` |
|
||||
|
|
|
|||
|
|
@ -208,9 +208,9 @@ KTX writes project state as plain files so agents can inspect and edit changes i
|
|||
|------|------------|---------|
|
||||
| `ktx.yaml` | `ktx setup` | Main project configuration: connections, LLM settings, embeddings, and context sources |
|
||||
| `.ktx/secrets/*` | `ktx setup` when file-backed secrets are selected | Local secret files referenced from `ktx.yaml`; do not commit these |
|
||||
| `semantic-layer/<connection-id>/*.yaml` | context build, ingestion, or `ktx sl write` | Semantic source definitions agents use for SQL generation |
|
||||
| `knowledge/global/*.md` | ingestion or `ktx wiki write --scope global` | Shared business context and metric definitions |
|
||||
| `knowledge/user/<user-id>/*.md` | `ktx wiki write --scope user` | User-scoped notes for one agent/user context |
|
||||
| `semantic-layer/<connection-id>/*.yaml` | context build, ingestion, or direct file edits | Semantic source definitions agents use for SQL generation |
|
||||
| `knowledge/global/*.md` | ingestion, memory capture, or direct file edits | Shared business context and metric definitions |
|
||||
| `knowledge/user/<user-id>/*.md` | memory capture or direct file edits | User-scoped notes for one agent/user context |
|
||||
| `.claude/skills/ktx/SKILL.md`, `.agents/skills/ktx/SKILL.md` | CLI-mode agent integration setup | Agent instructions for calling public `ktx` commands |
|
||||
|
||||
## Verify it worked
|
||||
|
|
|
|||
|
|
@ -26,10 +26,7 @@ ktx status --json
|
|||
# List sources
|
||||
ktx sl list --json
|
||||
ktx sl list --json --connection-id my-postgres
|
||||
ktx sl list --json --query "revenue"
|
||||
|
||||
# Read a source
|
||||
ktx sl read orders --json --connection-id my-postgres
|
||||
ktx sl search "revenue" --json
|
||||
|
||||
# Run a query from a JSON file
|
||||
ktx sl query --json \
|
||||
|
|
@ -44,9 +41,6 @@ ktx sl query --json \
|
|||
```bash
|
||||
# Search knowledge pages
|
||||
ktx wiki search "revenue recognition" --json --limit 10
|
||||
|
||||
# Read a specific page
|
||||
ktx wiki read order-status-definitions --json
|
||||
```
|
||||
|
||||
## Setting Up Your Agent
|
||||
|
|
|
|||
|
|
@ -10,11 +10,11 @@ After building context through scanning and ingestion, you'll want to refine it
|
|||
Agents should refine context in this order:
|
||||
|
||||
1. `ktx sl list --json` — discover available sources and connection ids.
|
||||
2. `ktx sl read <source> --connection-id <id>` — inspect the current YAML.
|
||||
3. Edit the source YAML directly or use `ktx sl write`.
|
||||
2. `ktx sl search <query> --json` — find source candidates for a concept.
|
||||
3. Edit the source YAML directly in `semantic-layer/<connection-id>/`.
|
||||
4. `ktx sl validate <source> --connection-id <id>` — verify columns, joins, and table references.
|
||||
5. `ktx sl query ... --format sql` — compile a representative query without executing it.
|
||||
6. `ktx wiki search ...` and `ktx wiki write ...` — add business context that does not belong in schema YAML.
|
||||
6. `ktx wiki search ...` — check business context captured by ingest or memory.
|
||||
|
||||
## Semantic Sources
|
||||
|
||||
|
|
@ -33,13 +33,14 @@ ktx sl list --connection-id my-postgres
|
|||
ktx sl list --json
|
||||
```
|
||||
|
||||
### Reading a source
|
||||
### Searching sources
|
||||
|
||||
```bash
|
||||
ktx sl read orders --connection-id my-postgres
|
||||
ktx sl search "revenue" --connection-id my-postgres --json
|
||||
```
|
||||
|
||||
This prints the full YAML definition for the source.
|
||||
Search returns ranked source summaries. To inspect or edit a source, open the
|
||||
YAML file under `semantic-layer/<connection-id>/`.
|
||||
|
||||
### The source schema
|
||||
|
||||
|
|
@ -147,25 +148,10 @@ Column visibility controls what agents see:
|
|||
| `internal` | Available for joins and measures but not shown to agents |
|
||||
| `hidden` | Excluded entirely — useful for ETL columns |
|
||||
|
||||
### Writing a source
|
||||
### Editing a source
|
||||
|
||||
```bash
|
||||
ktx sl write orders --connection-id my-postgres --yaml '
|
||||
name: orders
|
||||
table: public.orders
|
||||
grain: [order_id]
|
||||
columns:
|
||||
- name: order_id
|
||||
type: string
|
||||
- name: total_amount
|
||||
type: number
|
||||
measures:
|
||||
- name: total_revenue
|
||||
expr: SUM(total_amount)
|
||||
'
|
||||
```
|
||||
|
||||
You can also edit source files directly — they live at `semantic-layer/<connection-id>/<source-name>.yaml` in your project directory.
|
||||
Edit source files directly. They live at
|
||||
`semantic-layer/<connection-id>/<source-name>.yaml` in your project directory.
|
||||
|
||||
### Validating sources
|
||||
|
||||
|
|
@ -225,11 +211,10 @@ The query planner is grain-aware — it understands the cardinality of joins and
|
|||
|
||||
### Workflow: edit and validate a source
|
||||
|
||||
1. `ktx sl read orders --connection-id my-postgres > /tmp/orders.yaml` — capture the current definition.
|
||||
2. Edit `/tmp/orders.yaml` to add columns, measures, joins, or descriptions.
|
||||
3. `ktx sl write orders --connection-id my-postgres --yaml "$(cat /tmp/orders.yaml)"` — write the updated source.
|
||||
4. `ktx sl validate orders --connection-id my-postgres` — check the definition against the live schema.
|
||||
5. `ktx sl query --connection-id my-postgres --measure total_revenue --dimension order_date --format sql` — compile a representative query.
|
||||
1. Open `semantic-layer/my-postgres/orders.yaml`.
|
||||
2. Edit the file to add columns, measures, joins, or descriptions.
|
||||
3. `ktx sl validate orders --connection-id my-postgres` — check the definition against the live schema.
|
||||
4. `ktx sl query --connection-id my-postgres --measure total_revenue --dimension order_date --format sql` — compile a representative query.
|
||||
|
||||
If validation fails, fix the YAML before asking an agent to use the source. Common validation failures are missing columns, invalid join targets, and measure expressions that reference fields outside the source.
|
||||
|
||||
|
|
@ -260,42 +245,16 @@ knowledge/
|
|||
- **Global pages** apply across all connections — business definitions, metric standards, company terminology.
|
||||
- **User-scoped pages** are private to a user ID — personal notes, local gotchas, or context you do not want shared globally.
|
||||
|
||||
### Writing pages
|
||||
### Editing pages
|
||||
|
||||
```bash
|
||||
ktx wiki write order-status-definitions \
|
||||
--scope global \
|
||||
--summary "Business definitions for order status values" \
|
||||
--content "## Order Statuses
|
||||
|
||||
- **pending**: Order placed but not yet processed
|
||||
- **confirmed**: Payment received, awaiting fulfillment
|
||||
- **shipped**: Order dispatched to carrier
|
||||
- **delivered**: Order received by customer
|
||||
- **cancelled**: Order cancelled before shipment
|
||||
|
||||
Orders in pending status for more than 48 hours are flagged for review." \
|
||||
--tag orders \
|
||||
--tag definitions \
|
||||
--sl-ref orders
|
||||
```
|
||||
|
||||
Write flags:
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--scope <scope>` | `global` (default) or `user` |
|
||||
| `--summary <text>` | Short description for search results (required) |
|
||||
| `--content <text>` | Full Markdown content (required) |
|
||||
| `--tag <tag>` | Categorization tag (repeatable) |
|
||||
| `--ref <ref>` | Reference to external resources (repeatable) |
|
||||
| `--sl-ref <ref>` | Link to a semantic source (repeatable) |
|
||||
Create and edit knowledge pages directly as Markdown files in the `knowledge/`
|
||||
directory. Ingest and memory capture also create these pages automatically.
|
||||
|
||||
Knowledge page fields:
|
||||
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| Key | Yes | Stable page identifier passed to `ktx wiki read` |
|
||||
| Key | Yes | Stable page identifier used as the Markdown filename |
|
||||
| Summary | Yes | Short text shown in search results |
|
||||
| Content | Yes | Full Markdown business context |
|
||||
| Scope | No | `global` for shared context or `user` for user-scoped notes |
|
||||
|
|
@ -303,20 +262,12 @@ Knowledge page fields:
|
|||
| External refs | No | Links or identifiers for source-of-truth systems |
|
||||
| Semantic-layer refs | No | Source names the page explains or constrains |
|
||||
|
||||
You can also create and edit knowledge pages directly as Markdown files in the `knowledge/` directory.
|
||||
|
||||
### Listing pages
|
||||
|
||||
```bash
|
||||
ktx wiki list
|
||||
```
|
||||
|
||||
### Reading a page
|
||||
|
||||
```bash
|
||||
ktx wiki read order-status-definitions
|
||||
```
|
||||
|
||||
### Searching
|
||||
|
||||
```bash
|
||||
|
|
@ -328,9 +279,9 @@ Search uses both full-text matching and semantic similarity — it finds relevan
|
|||
### Workflow: add searchable business context
|
||||
|
||||
1. Search first: `ktx wiki search "order status definitions"`.
|
||||
2. If no page already covers the rule, write a page with `ktx wiki write`.
|
||||
3. Include a concise `--summary`; agents see this before loading full content.
|
||||
4. Add `--tag` values for the business area and `--sl-ref` values for related semantic sources.
|
||||
2. If no page already covers the rule, create or edit a Markdown file under `knowledge/global/`.
|
||||
3. Include concise frontmatter; agents see the summary before loading full content.
|
||||
4. Add `tags` values for the business area and `sl_refs` values for related semantic sources.
|
||||
5. Search again with the user's likely wording to confirm the page is discoverable.
|
||||
|
||||
## Common errors
|
||||
|
|
@ -341,4 +292,4 @@ Search uses both full-text matching and semantic similarity — it finds relevan
|
|||
| Query compilation double-counts a measure | Join relationship or grain is missing or wrong | Add `grain` and explicit `relationship` values, then validate and recompile |
|
||||
| Agent cannot find a metric | Measure name or description does not match business terminology | Add a measure description and a knowledge page with common synonyms |
|
||||
| Knowledge search misses a page | Summary and tags do not include likely user wording | Rewrite the summary and add relevant tags, then search again |
|
||||
| `ktx sl write` changes are hard to review | Large YAML was passed inline | Edit the source file directly or write from a temporary file, then review the git diff |
|
||||
| Semantic-layer changes are hard to review | The YAML edit is too large or unfocused | Split the change into smaller source-file edits, then review the git diff |
|
||||
|
|
|
|||
|
|
@ -34,11 +34,9 @@ description: Use local KTX semantic context and wiki knowledge for this project.
|
|||
Available commands:
|
||||
- `ktx status --json --project-dir /path/to/project`
|
||||
- `ktx sl list --json --project-dir /path/to/project`
|
||||
- `ktx sl list --json --project-dir /path/to/project --query '<text>'`
|
||||
- `ktx sl read '<sourceName>' --json --project-dir /path/to/project --connection-id '<id>'`
|
||||
- `ktx sl search '<text>' --json --project-dir /path/to/project --connection-id '<id>'`
|
||||
- `ktx sl query --json --project-dir /path/to/project --connection-id '<id>' --query-file '<path>' --execute --max-rows 100`
|
||||
- `ktx wiki search '<query>' --json --project-dir /path/to/project --limit 10`
|
||||
- `ktx wiki read '<pageId>' --json --project-dir /path/to/project`
|
||||
```
|
||||
|
||||
### Workflow tips
|
||||
|
|
@ -127,12 +125,8 @@ All supported agent clients call the same KTX CLI commands:
|
|||
|---------|-------------|
|
||||
| `ktx status --json` | Return project setup and context readiness |
|
||||
| `ktx wiki search <query> --json` | Search knowledge pages |
|
||||
| `ktx wiki read <key> --json` | Read a knowledge page |
|
||||
| `ktx wiki write <key>` | Write or update a knowledge page |
|
||||
| `ktx sl list --json` | List semantic-layer sources |
|
||||
| `ktx sl list --query <text> --json` | Search semantic-layer sources |
|
||||
| `ktx sl read <source> --json --connection-id <id>` | Read a semantic source definition |
|
||||
| `ktx sl write <source> --connection-id <id>` | Write or update a semantic source |
|
||||
| `ktx sl search <query> --json` | Search semantic-layer sources |
|
||||
| `ktx sl validate <source> --connection-id <id>` | Validate semantic source definitions |
|
||||
| `ktx sl query --json` | Execute a semantic-layer query when semantic compute is configured |
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue