apunkt/ktx
1
0
Fork 0
mirror of https://github.com/Kaelio/ktx.git synced 2026-06-07 07:55:13 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat(setup): add Claude Desktop target and MCP-first agent setup (#114)

Browse source 
* feat(setup): add Claude Desktop target and MCP-first agent setup

Adds `ktx mcp stdio` and a `claude-desktop` setup target that generates a
local plugin ZIP wiring the analytics skill and a stdio MCP config. Replaces
the CLI-only agent install mode with MCP+analytics (default) and an optional
admin CLI skill, renames the research skill to analytics, and lets interactive
setup pick project vs global scope when every target supports it. Extracts a
shared MCP server factory used by both HTTP and stdio entrypoints.

* Add MCP agent client setup support

* Polish setup output formatting

* Add MCP tool polish design spec

Design for slimming the MCP-registered surface from 25 to 11 tools,
introducing memory_ingest, applying the per-tool polish kit (annotations,
outputSchema, .describe(), in-band error wrapping, union-drift fixes,
type-narrowed jsonToolResult), emitting progress notifications on
sql_execution + sl_query, and refining the ktx-analytics SKILL.md to
match.

* Refine MCP tool polish design spec after adversarial review iteration 1

* Refine MCP tool polish design spec after adversarial review iteration 2

* Refine MCP tool polish design spec after adversarial review iteration 3

* refactor(context): rename memory capture service to ingest

* feat(mcp): slim research tool surface

* refactor(mcp): remove admin ports from server factory

* refactor(cli): rename text ingest memory port

* docs: update analytics skill for memory ingest

* chore: verify mcp surface rename

* Add MCP tool polish v1 surface change plan

* feat(context): polish mcp tool metadata

* fix(context): enforce resolved semantic layer compute sources

* feat(context): emit mcp query progress stages

* fix(context): keep mcp progress event internal

* Add MCP tool polish v1 metadata & progress plan

* Fix CI snapshot and docs checks
This commit is contained in:
Andrey Avtomonov 2026-05-16 11:39:55 +02:00 • committed by GitHub
parent a72fca2b32
commit e6d578c03f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
50 changed files with 8092 additions and 3143 deletions
Download patch file Download diff file Expand all files Collapse all files

60
README.md
View file

@ -6,6 +6,8 @@
The context layer for analytics agents
</h1>
<p align="center">by Kaelio</p>
<p align="center">
<a href="https://www.npmjs.com/package/@kaelio/ktx"><img src="https://img.shields.io/npm/v/@kaelio/ktx?style=flat-square&color=f97316" alt="npm version" /></a>
<a href="https://codecov.io/gh/Kaelio/ktx"><img src="https://codecov.io/gh/Kaelio/ktx/branch/main/graph/badge.svg" alt="Codecov" /></a>
@ -32,7 +34,7 @@ SQLite.
## Quick Start
```bash
pnpm add --global @kaelio/ktx
npm install -g @kaelio/ktx
ktx setup
ktx status
```
@ -40,6 +42,19 @@ ktx status
`ktx setup` creates or resumes a local KTX project, configures providers and
connections, builds context, and installs agent integration.
Example `ktx status` output after setup:
```text
KTX project: /home/user/analytics
Project ready: yes
LLM ready: yes (claude-sonnet-4-6)
Embeddings ready: yes (text-embedding-3-small)
Databases configured: yes (postgres-warehouse)
Context sources configured: yes (dbt-main)
KTX context built: yes
Agent integration ready: yes (codex:project)
```
## Common Commands
| Command | Purpose |
@ -92,6 +107,49 @@ ktx wiki search "refund policy" --json
ktx sl query --connection-id warehouse --measure orders.revenue --format sql
```
During agent setup, choose **MCP tools + analytics skill** for client agents.
Choose **MCP tools + analytics skill + admin CLI skill** only when a developer
or operator agent also needs pinned `ktx` admin commands.
The analytics skill teaches client agents the MCP workflow: discover data,
prefer semantic-layer measures, inspect entity details before raw SQL, and
capture durable learnings. Admin CLI skills call `ktx` commands directly
through a skill file installed in your agent's config:
```bash
ktx sl query --measure orders.revenue --dimension orders.status --format sql
ktx wiki search "revenue definition"
ktx sl validate orders
```
Supported client agents: Claude Code, Claude Desktop, Codex, Cursor, OpenCode,
and clients that can use the printed MCP endpoint or `.agents` admin skills.
Claude Desktop setup registers a local `ktx mcp stdio` server in Claude
Desktop's config and generates `.ktx/agents/claude/ktx-plugin.zip` with the
analytics skill.
The release artifact manifest contains the public npm tarball and the bundled
`kaelio-ktx` runtime wheel. The `python/ktx-sl` and `python/ktx-daemon`
directories remain source packages for development, not public release
artifacts.
## Workspace packages
| Package | Purpose |
|---------|---------|
| `packages/cli` | CLI entry point |
| `packages/context` | Core context engine |
| `packages/llm` | LLM and embedding providers |
| `packages/connector-bigquery` | BigQuery scan connector |
| `packages/connector-clickhouse` | ClickHouse scan connector |
| `packages/connector-mysql` | MySQL scan connector |
| `packages/connector-postgres` | Postgres scan connector |
| `packages/connector-snowflake` | Snowflake scan connector |
| `packages/connector-sqlite` | SQLite scan connector |
| `packages/connector-sqlserver` | SQL Server scan connector |
| `python/ktx-sl` | Semantic-layer query planning |
| `python/ktx-daemon` | Portable compute service |
## Development
```bash

164
docs-site/content/docs/integrations/agent-clients.mdx
View file

@ -1,18 +1,26 @@
---
title: Agent Clients
description: Set up KTX with Claude Code, Cursor, Codex, and OpenCode.
description: Set up KTX with Claude Code, Claude Desktop, Cursor, Codex, and OpenCode.
---
KTX integrates with coding agents through CLI skills and command files. These
files teach agents to call public `ktx` commands directly from the terminal for
semantic-layer context and wiki knowledge.
KTX exposes context to end-user agents through MCP tools. The CLI remains the
admin surface for setup, ingest, status, daemon lifecycle, and debugging.
Run `ktx setup` and select your agent targets, or configure manually using the
snippets below. Setup pins generated skill files to the KTX CLI path that
created them, so agents do not need `ktx` on `PATH`.
Run `ktx setup` and select your client agent targets, or configure manually
using the snippets below. Choose **MCP tools + analytics skill** for client
agents. Choose **MCP tools + analytics skill + admin CLI skill** only when a
developer or operator agent also needs pinned `ktx` admin commands.
## Install with setup
Start the MCP server before connecting an end-user agent:
```bash
ktx mcp start
```
Then install client integration:
```bash
ktx setup --agents
```
@ -23,7 +31,8 @@ Use `--target` for one target:
ktx setup --agents --target codex
```
Use `--global` only with `claude-code` or `codex`:
Use `--global` only with `claude-code` or `codex`. Claude Desktop always
generates a project-local plugin ZIP:
```bash
ktx setup --agents --target claude-code --global
@ -34,19 +43,54 @@ KTX records installed files in `.ktx/agents/install-manifest.json`. That
manifest lets status checks report agent readiness and lets future cleanup
remove only files KTX installed.
The interactive command asks two questions:
```txt
◆ How should client agents connect to this KTX project?
│ ○ MCP tools + analytics skill
│ ○ MCP tools + analytics skill + admin CLI skill
└
◆ Which agent targets should KTX install?
│ ◻ Claude Code
│ ◻ Claude Desktop
│ ◻ Codex
│ ◻ Cursor
│ ◻ OpenCode
│ ◻ Universal .agents
└
```
When every selected target supports both project and global setup, the command
also asks where to install supported agent config:
```txt
◆ Where should KTX install supported agent config?
│ ○ Project
│ ○ Global
└
```
## Generated files
| Target | Project-scoped files | Global files |
|--------|----------------------|--------------|
| Claude Code | `.claude/skills/ktx/SKILL.md`, `.claude/rules/ktx.md` | `~/.claude/skills/ktx/SKILL.md`, `~/.claude/rules/ktx.md` |
| Codex | `.agents/skills/ktx/SKILL.md`, `.codex/instructions/ktx.md` | `$CODEX_HOME/skills/ktx/SKILL.md`, `$CODEX_HOME/instructions/ktx.md` |
| Cursor | `.cursor/rules/ktx.mdc` | Not supported |
| OpenCode | `.opencode/commands/ktx.md` | Not supported |
| Universal `.agents` | `.agents/skills/ktx/SKILL.md` | Not supported |
KTX writes MCP client configuration and an analytics skill by default. It writes
admin CLI skills only when you choose **MCP tools + analytics skill + admin CLI
skill**.
Skill files list pinned `ktx` commands. Rule files tell the agent when KTX is
appropriate, such as data schemas, metrics, dimensions, database structure, and
SQL questions.
| Target | MCP tools + analytics skill | Adds with admin CLI skill |
|--------|------------------------------|---------------------------|
| Claude Code | `.mcp.json`, `.claude/skills/ktx-analytics/SKILL.md` | `.claude/skills/ktx/SKILL.md`, `.claude/rules/ktx.md` |
| Claude Desktop | `~/Library/Application Support/Claude/claude_desktop_config.json` stdio entry + `.ktx/agents/claude/ktx-plugin.zip` with analytics skill | Adds `skills/ktx/SKILL.md` inside the plugin ZIP |
| Codex | Printed snippet for `~/.codex/config.toml`, `.agents/skills/ktx-analytics/SKILL.md` | `.agents/skills/ktx/SKILL.md`, `.codex/instructions/ktx.md` |
| Cursor | `.cursor/mcp.json`, `.cursor/rules/ktx-analytics.mdc` | `.cursor/rules/ktx.mdc` |
| OpenCode | Printed snippet for `opencode.json`, `.opencode/commands/ktx-analytics.md` | `.opencode/commands/ktx.md` |
| Universal `.agents` | Printed MCP endpoint, `.agents/skills/ktx-analytics/SKILL.md` | `.agents/skills/ktx/SKILL.md` |
MCP config gives agents access to KTX context tools such as discovery,
semantic-layer queries, wiki search, SQL execution, and memory ingest. The
analytics skill explains how to use those tools for semantic-layer-first
analysis. Optional admin skill and rule files list pinned CLI commands for
developer or operator agents.
## Claude Code
@ -56,13 +100,16 @@ During setup, select **Claude Code** from the agent targets. KTX writes:
| Scope | Files |
|-------|-------|
| Project | `.claude/skills/ktx/SKILL.md`, `.claude/rules/ktx.md` |
| Global | `~/.claude/skills/ktx/SKILL.md`, `~/.claude/rules/ktx.md` |
| Project | `.mcp.json`, `.claude/skills/ktx-analytics/SKILL.md`; optional `.claude/skills/ktx/SKILL.md`, `.claude/rules/ktx.md` |
| Global | `~/.claude.json`, `~/.claude/skills/ktx-analytics/SKILL.md`; optional `~/.claude/skills/ktx/SKILL.md`, `~/.claude/rules/ktx.md` |
Both project-scoped and global installations are supported.
### Manual CLI skills configuration
Use manual CLI skills only for developer or operator agents that need admin
commands. End-user data agents use MCP.
Create `.claude/skills/ktx/SKILL.md`:
```markdown title=".claude/skills/ktx/SKILL.md"
@ -82,6 +129,7 @@ Available commands:
### Workflow tips
- Claude Code discovers skills automatically from `.claude/skills/`.
- Claude Code reads MCP config from `.mcp.json` for project-scoped MCP tools.
- Claude rules in `.claude/rules/` tell Claude when KTX should be used.
- Global installation makes KTX available in all projects without per-project setup.
- Keep generated skills committed only when your team wants project-local agent instructions in git.
@ -96,13 +144,19 @@ During setup, select **Cursor** from the agent targets. KTX writes:
| Mode | File |
|------|------|
| CLI rules | `.cursor/rules/ktx.mdc` |
| MCP tools + analytics skill | `.cursor/mcp.json`, `.cursor/rules/ktx-analytics.mdc` |
| Admin CLI rules | `.cursor/rules/ktx.mdc` |
Cursor supports project-scoped installation only.
### Manual CLI rules configuration
Create `.cursor/rules/ktx.mdc` with the same content structure as the Claude Code `SKILL.md` file. Cursor rules use the `.mdc` extension but support the same markdown command definitions.
Use manual CLI rules only for developer or operator agents that need admin
commands. End-user data agents use MCP.
Create `.cursor/rules/ktx.mdc` with the same content structure as the Claude
Code `SKILL.md` file. Cursor rules use the `.mdc` extension but support the
same markdown command definitions.
### Workflow tips
@ -111,6 +165,37 @@ Create `.cursor/rules/ktx.mdc` with the same content structure as the Claude Cod
---
## Claude Desktop
During setup, select **Claude Desktop** from the agent targets. KTX writes the
MCP server entry directly into Claude Desktop's config and generates a separate
plugin ZIP for the analytics skill:
- `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or
`%AppData%/Claude/claude_desktop_config.json` (Windows) gets an
`mcpServers.ktx` entry that runs the KTX MCP server over stdio via a local
launcher shim at `.ktx/agents/claude/ktx-plugin-runner.sh`. The shim locates
a usable Node.js (Volta, NVM, Homebrew, system) so Claude Desktop can spawn
the server without needing `node` in PATH.
- `.ktx/agents/claude/ktx-plugin.zip` contains the `ktx-analytics` skill (and
the admin `ktx` skill if you choose **MCP tools + analytics skill + admin
CLI skill**). Install the ZIP from Claude Desktop's plugin UI to load the
skill.
After `ktx setup`, restart Claude Desktop so it picks up the new MCP server
entry, then install the plugin ZIP. No daemon needs to be running — Claude
Desktop spawns the MCP server itself per session.
Claude Desktop does not introspect local stdio MCP servers, so the per-tool
"Connector"-style UI is not rendered for KTX. The tools are still callable
from any Claude Desktop chat.
If you move the KTX checkout or project directory, rerun `ktx setup --agents`
to refresh the absolute paths in `claude_desktop_config.json` and the launcher
shim, then reinstall the regenerated plugin ZIP.
---
## Codex
### Install via `ktx setup`
@ -119,15 +204,19 @@ During setup, select **Codex** from the agent targets. KTX writes:
| Scope | Files |
|-------|-------|
| Project | `.agents/skills/ktx/SKILL.md`, `.codex/instructions/ktx.md` |
| Global | `$CODEX_HOME/skills/ktx/SKILL.md`, `$CODEX_HOME/instructions/ktx.md` |
| Project | MCP snippet, `.agents/skills/ktx-analytics/SKILL.md`; optional `.agents/skills/ktx/SKILL.md`, `.codex/instructions/ktx.md` |
| Global | MCP snippet, `$CODEX_HOME/skills/ktx-analytics/SKILL.md`; optional `$CODEX_HOME/skills/ktx/SKILL.md`, `$CODEX_HOME/instructions/ktx.md` |
Both project-scoped and global installations are supported. `CODEX_HOME`
defaults to `~/.codex`.
### Manual CLI skills configuration
Create `.agents/skills/ktx/SKILL.md` with the same content structure as Claude Code's `SKILL.md`.
Use manual CLI skills only for developer or operator agents that need admin
commands. End-user data agents use MCP.
Create `.agents/skills/ktx/SKILL.md` with the same content structure as Claude
Code's `SKILL.md`.
### Workflow tips
@ -146,13 +235,18 @@ During setup, select **OpenCode** from the agent targets. KTX writes:
| Mode | File |
|------|------|
| CLI commands | `.opencode/commands/ktx.md` |
| MCP tools + analytics skill | Snippet for `opencode.json`, `.opencode/commands/ktx-analytics.md` |
| Admin CLI commands | `.opencode/commands/ktx.md` |
OpenCode supports project-scoped installation only.
### Manual CLI commands configuration
Create `.opencode/commands/ktx.md` with the same command definitions as Claude Code's `SKILL.md`.
Use manual CLI commands only for developer or operator agents that need admin
commands. End-user data agents use MCP.
Create `.opencode/commands/ktx.md` with the same command definitions as Claude
Code's `SKILL.md`.
### Workflow tips
@ -163,7 +257,7 @@ Create `.opencode/commands/ktx.md` with the same command definitions as Claude C
## Command reference
All supported agent clients call the same KTX CLI commands:
Admin CLI skills call the same KTX CLI commands:
| Command | Description |
|---------|-------------|
@ -183,9 +277,11 @@ All supported agent clients call the same KTX CLI commands:
## Comparison
| | Claude Code | Cursor | Codex | OpenCode |
|---|---|---|---|---|
| CLI skills | Yes | Yes (.mdc) | Yes | Yes |
| Global install | Yes | No | Yes | No |
| Rule or instruction file | `.claude/rules/ktx.md` | `.cursor/rules/ktx.mdc` | `.codex/instructions/ktx.md` | `.opencode/commands/ktx.md` |
| Skill file | `.claude/skills/ktx/SKILL.md` | Not separate | `.agents/skills/ktx/SKILL.md` | Not separate |
| | Claude Code | Claude Desktop | Cursor | Codex | OpenCode |
|---|---|---|---|---|---|
| MCP tools | Yes | Local stdio via `claude_desktop_config.json` | Yes | Snippet | Snippet |
| Analytics skill | `.claude/skills/ktx-analytics/SKILL.md` | Included in plugin ZIP | `.cursor/rules/ktx-analytics.mdc` | `.agents/skills/ktx-analytics/SKILL.md` | `.opencode/commands/ktx-analytics.md` |
| Admin CLI skills | Optional | Optional in plugin ZIP | Optional (.mdc) | Optional | Optional |
| Global install | Yes | Project-local ZIP | No | Yes | No |
| Rule or instruction file | `.claude/rules/ktx.md` | Plugin `SETUP.md` | `.cursor/rules/ktx.mdc` | `.codex/instructions/ktx.md` | `.opencode/commands/ktx.md` |
| Skill file | `.claude/skills/ktx/SKILL.md` | `skills/ktx/SKILL.md` in plugin ZIP | Not separate | `.agents/skills/ktx/SKILL.md` | Not separate |

2
docs-site/next-env.d.ts vendored
View file

@ -1,6 +1,6 @@
/// <reference types="next" />
/// <reference types="next/image-types/global" />
import "./.next/dev/types/routes.d.ts";
import "./.next/types/routes.d.ts";
// NOTE: This file should not be edited
// see https://nextjs.org/docs/app/api-reference/config/typescript for more information.

18
docs/superpowers/plans/2026-05-14-research-agent-mcp-discover-data.md
View file

@ -903,14 +903,16 @@ Add this test after the `dictionary_search` registration test:
limit: 5,
}),
).resolves.toMatchObject({
structuredContent: [
{
kind: 'table',
id: 'public.orders',
connectionId: 'warehouse',
tableRef: { catalog: null, db: 'public', name: 'orders' },
},
],
structuredContent: {
refs: [
{
kind: 'table',
id: 'public.orders',
connectionId: 'warehouse',
tableRef: { catalog: null, db: 'public', name: 'orders' },
},
],
},
});
expect(discover.search).toHaveBeenCalledWith({
query: 'orders',

1459
docs/superpowers/plans/2026-05-16-mcp-tool-polish-v1-metadata-progress.md Normal file
View file

File diff suppressed because it is too large Load diff

1305
docs/superpowers/plans/2026-05-16-mcp-tool-polish-v1-surface-change.md Normal file
View file

File diff suppressed because it is too large Load diff

4
docs/superpowers/specs/2026-05-14-research-agent-mcp-tools-design.md
View file

@ -143,7 +143,9 @@ or `entity_details` tools.
}
```
**Output:** array of refs, each:
**Output:** `{ refs: Ref[] }` — the MCP protocol requires `structuredContent`
to be a JSON object, so the array of matches is wrapped under `refs`. Each
ref is shaped:
```typescript
{

802
docs/superpowers/specs/2026-05-16-mcp-tool-polish-design.md Normal file
View file

@ -0,0 +1,802 @@
# MCP Tool Polish: Slim Research Surface + Spec Compliance
**Date:** 2026-05-16
**Author:** Andrey Avtomonov
**Status:** Design — pending implementation plan
## Background
KTX currently exposes 25 MCP tools across context, semantic-layer, ingest, and
scan ports (`packages/context/src/mcp/context-tools.ts`). The
`ktx-analytics` SKILL.md (`packages/cli/src/skills/analytics/SKILL.md`)
is installed into every supported MCP client by `ktx setup --agents` and
already describes a Douala-equivalent research methodology (Discover → Inspect
→ Resolve → Plan → Query → Validate → Capture) that references nine of those
tools.
A recent in-session audit surfaced three real bug classes:
- **`structuredContent` shape** — `discover_data` returned a bare array; MCP
requires an object. Fixed.
- **Union-shape LLM drift** — `sl_query.order_by` accepted
`{ field, direction }` but Claude emitted Cube-style `{ id, desc }`. Fixed
via a `z.preprocess` that normalizes the alt-shape before strict validation.
- **Contract leak to the Python daemon** — `compileLocalSlQuery` skipped
`toResolvedWire` and sent TS-only authoring fields (`usage`,
`inherits_columns_from`) to a Pydantic model with `extra="forbid"`. Fixed.
The audit also identified systemic gaps applicable across the surface: no
per-field `.describe()` outside `memory_capture*`, no `outputSchema` declared
anywhere, no MCP tool annotations, lingering union-drift risk on
`slQueryDimensionSchema` (`{ dimension, granularity }`) and
`entityDetailsTableRefSchema` (`{ schema, table }`), and inconsistent error
handling (only `sql_execution` wraps thrown errors in-band per MCP spec; the
other 24 let exceptions propagate as JSON-RPC errors).
The current MCP spec (2025-11-25) provides several mechanisms KTX does not
use: `outputSchema` with `structuredContent` validation, tool annotations
(`readOnlyHint`/`destructiveHint`/`idempotentHint`/`openWorldHint`/`title`),
progress notifications via `_meta.progressToken`, and a clear in-band error
contract (`isError: true` with text content).
The 25-tool surface is also wider than the agent needs. The `ktx-analytics`
SKILL only orchestrates nine of them; the rest are admin/setup/maintenance
operations that are better served by a `ktx`-CLI flow (and a future
`ktx-admin` SKILL — out of scope for this spec).
## Goals
- Reduce the MCP-registered surface to **11 tools** focused on the research
loop: `connection_list`, `discover_data`, `wiki_search`, `wiki_read`,
`entity_details`, `dictionary_search`, `sl_read_source`, `sl_query`,
`sql_execution`, `memory_ingest` (new), `memory_ingest_status` (new). The
remaining 14 tools become CLI-only by removing their MCP registration; their
implementations stay in `packages/context/src/` to back the CLI.
- Replace `memory_capture` / `memory_capture_status` with `memory_ingest` /
`memory_ingest_status`. The new tool takes free-form markdown `content`
plus optional `connectionId`; the memory agent triages into wiki and SL as
before.
- Apply the per-tool polish kit on every retained tool: MCP tool annotations,
`outputSchema`, per-field `.describe()`, rewritten tool descriptions,
standardized in-band error handling, union-drift normalization on the two
remaining at-risk schemas, and a type-narrowed `jsonToolResult`.
- Emit MCP progress notifications from `sql_execution` and `sl_query`.
- Update `ktx-analytics` SKILL.md to use the renamed tool, broaden the capture
step, and document multi-connection routing.
## Non-Goals
- Admin CLI skill (separate spec).
- Deleting the source code of the admin tools (deferred follow-up gated by
the admin CLI skill landing).
- MCP resources (subscribable wiki / SL).
- MCP prompts pushed by the server (the analytics SKILL is the equivalent).
- Elicitation, sampling, tool icons.
- A code-execution tool / Python sandbox (separate spec; the analytics
workflow does not require one for the goals above).
- Per-client schema-feature workarounds beyond what the audit findings
already cover. Codex's no-header-auth limitation is unrelated to tool
shape and is left to `setup-agents.ts` to document.
- Multi-tenancy, telemetry, rate limiting.
## Design
### 1. Surface change
#### 1.1 Retained tools (11)
| # | Tool | Port |
|---|---|---|
| 1 | `connection_list` | `KtxConnectionsMcpPort.list` |
| 2 | `discover_data` | `KtxDiscoverDataMcpPort.search` |
| 3 | `wiki_search` | `KtxKnowledgeMcpPort.search` |
| 4 | `wiki_read` | `KtxKnowledgeMcpPort.read` |
| 5 | `entity_details` | `KtxEntityDetailsMcpPort.read` |
| 6 | `dictionary_search` | `KtxDictionarySearchMcpPort.search` |
| 7 | `sl_read_source` | `KtxSemanticLayerMcpPort.readSource` |
| 8 | `sl_query` | `KtxSemanticLayerMcpPort.query` |
| 9 | `sql_execution` | `KtxSqlExecutionMcpPort.execute` |
| 10 | `memory_ingest` | New port `KtxMemoryIngestMcpPort.ingest` |
| 11 | `memory_ingest_status` | `KtxMemoryIngestMcpPort.status` |
`connection_list` is retained because in multi-connection projects, the agent
needs a way to enumerate available connections before issuing a `sql_execution`
or `sl_query` against a specific one.
**`connectionId` resolution per retained tool** (auto-resolution exists today
only on the local SL path; do not broaden it as part of this spec):
| Tool | `connectionId` | Auto-resolves to single connection if omitted? |
|---|---|---|
| `connection_list` | n/a | n/a |
| `discover_data` | optional | no — search is run unscoped when omitted |
| `wiki_search` | n/a | n/a |
| `wiki_read` | n/a | n/a |
| `entity_details` | required | no |
| `dictionary_search` | optional | no — search is run unscoped when omitted |
| `sl_read_source` | required | no |
| `sl_query` | optional | yes — `resolveLocalConnectionId` (`packages/context/src/sl/local-query.ts`) auto-resolves when the project has exactly one connection |
| `sql_execution` | required | no |
| `memory_ingest` | optional | no — omitted means "global" knowledge (wiki only — see below) |
| `memory_ingest_status` | n/a | n/a |
The skill update in §3 must reflect this matrix: when `connection_list` shows
multiple connections, the agent always passes `connectionId` for the required
tools and for `sl_query`/`discover_data`/`dictionary_search` whenever the user
intent pins a specific warehouse.
**`memory_ingest` connectionId semantics — important constraint.** The
underlying `MemoryAgentService.ingest` derives `hasSL = !!input.connectionId`
(`packages/context/src/memory/memory-agent.service.ts:55`) and only wires the
SL-capable toolset when `connectionId` is supplied
(`packages/context/src/memory/memory-agent.service.ts:116-118`). Therefore
`memory_ingest` can update the semantic layer **only** when `connectionId` is
provided. Omit `connectionId` only for genuinely global wiki-only knowledge
(company-wide policies, vocabulary, user preferences); supply `connectionId`
for any knowledge that touches a specific warehouse — including measure
definitions, schema gotchas, and any wording like "in our warehouse" or "this
warehouse". The §3 SKILL update and the worked example must enforce this.
#### 1.2 Removed from MCP registration
`connection_test`, `wiki_write`, `sl_list_sources`, `sl_write_source`,
`sl_validate`, `ingest_trigger`, `ingest_status`, `ingest_report`,
`ingest_replay`, `scan_trigger`, `scan_status`, `scan_report`,
`scan_list_artifacts`, `scan_read_artifact`,
plus `memory_capture` and `memory_capture_status` (replaced).
The conditional registration blocks in `registerKtxContextTools` for these
ports are removed. The underlying `KtxIngestMcpPort`, `KtxScanMcpPort`, etc.
implementations stay; the `ktx` CLI uses them directly. The `KtxMcpContextPorts`
type drops the removed `ingest?`, `scan?`, etc. fields. `MemoryCapturePort` is
renamed to `MemoryIngestPort`.
#### 1.3 New tool — `memory_ingest`
Replaces `memory_capture`. The change is a rename + a slightly relaxed input
contract; the underlying `MemoryCaptureService`
(`packages/context/src/memory/memory-runs.ts:81`) is reused as-is and renamed to
`MemoryIngestService`. No alias, no migration shim — per the standing
no-back-compat rule, the rename is atomic with the SKILL update.
**Final internal API shape after the rename — no compatibility wrappers:**
| Old name | New name |
|---|---|
| `MemoryCaptureService` (class) | `MemoryIngestService` |
| `MemoryCaptureService.capture(input)` (method) | `MemoryIngestService.ingest(input)` |
| `MemoryCaptureServiceDeps` | `MemoryIngestServiceDeps` |
| `MemoryCaptureStartResult` | `MemoryIngestStartResult` |
| `MemoryCaptureStatus` (return type) | `MemoryIngestStatus` |
| `MemoryCapturePort` (in `mcp/types.ts`) | `MemoryIngestPort` (with `.ingest()` and `.status()`) |
| `MemoryCapturePort.capture()` | `MemoryIngestPort.ingest()` |
| `TextMemoryCapturePort` (CLI, `text-ingest.ts`) | `TextMemoryIngestPort` (with `.ingest()`, `.waitForRun()`, `.status()`) |
| `createLocalProjectMemoryCapture` factory | `createLocalProjectMemoryIngest` |
Every internal call site (`packages/context/src/mcp/server.ts`,
`packages/context/src/mcp/local-project-ports.ts`,
`packages/cli/src/mcp-server-factory.ts`, `packages/cli/src/text-ingest.ts`,
their tests, and the `packages/context/src/memory/index.ts` re-exports) is
updated in lockstep. The agent-facing `MemoryAgentService.ingest` method and
its `MemoryAgentInput` type are unchanged.
**Mapping `memory_ingest` input → `MemoryAgentInput`** (defined in
`packages/context/src/memory/types.ts`):
| `MemoryAgentInput` field | Value supplied by `memory_ingest` handler |
|---|---|
| `userId` | `userContext.userId` (existing pattern) |
| `chatId` | `mcp-${randomUUID()}` (existing pattern) |
| `userMessage` | synthetic framing string, e.g. `Ingest external knowledge into KTX memory.` |
| `assistantMessage` | input.`content` |
| `connectionId` | input.`connectionId` (when provided) |
| `sourceType` | `'external_ingest'` |
The free-form markdown is routed into `assistantMessage` (not `userMessage`)
with a synthetic framing `userMessage`, mirroring the existing CLI text-ingest
path (`packages/cli/src/text-ingest.ts:295-302`). This mapping is required so
that `detectCaptureSignals`
(`packages/context/src/memory/capture-signals.ts:14`) can fire its
`assistantMessage`-keyed cues — SQL aggregates, LookML structure, and
definition tables — for artifact-like content. Routing content into
`userMessage` would lose those signals and silently degrade triage parity with
CLI ingest. The existing memory-agent prompt and tests already expect this
shape; no changes to the memory agent itself are required.
Acceptance criterion: an MCP `memory_ingest` call with the same markdown
content as a CLI `ktx ingest text` invocation must produce identical
`CaptureSignals` (knowledge / sl / dialect / reasons) — covered by a parity
test that feeds the same fixture content through both ingest entry points and
asserts equal `detectCaptureSignals` output.
**Input schema:**
```typescript
const memoryIngestSchema = z.object({
content: z
.string()
.min(1)
.describe(
'Free-form markdown to ingest. Include the knowledge itself plus any ' +
'context (source, the user\'s question, why this came up) that the ' +
'memory agent should consider when triaging into wiki/SL.',
),
connectionId: z
.string()
.min(1)
.optional()
.describe(
'Scope this memory to a specific connection. REQUIRED when the knowledge ' +
'is warehouse-specific (measure definitions, schema gotchas, anything ' +
'tied to a particular warehouse) — without it the memory agent cannot ' +
'update the semantic layer and the knowledge will land as wiki-only. ' +
'Omit only for genuinely global wiki knowledge (company-wide policies, ' +
'vocabulary, user preferences).',
),
});
```
**Tool description:**
> Ingest free-form knowledge into KTX's durable memory so it is available to
> future turns. Call this whenever a research turn produces something worth
> remembering — business rules, metric definitions, gotchas, schema
> explanations, recurring findings — **or** whenever the user asks you to
> remember something. Pass everything in `content` as markdown: the finding,
> plus any source or context that helps the memory agent triage. KTX's memory
> agent decides whether the content belongs in the wiki, the semantic layer,
> or both. Each call is a feedback loop — better notes here mean smarter
> `discover_data` and `wiki_search` results for everyone next time.
**Returns:** `{ runId: string }` — same shape as today's `memory_capture`.
**`memory_ingest_status`** mirrors today's `memory_capture_status` exactly,
renamed only.
### 2. Per-tool polish kit
#### 2.0 Registration topology — memory tools must share the polish path
Today `memory_capture` / `memory_capture_status` are registered in
`packages/context/src/mcp/server.ts` via direct `deps.server.registerTool`
calls (`server.ts:23,45`), **bypassing** `registerParsedTool` in
`context-tools.ts`. If left as-is, the polish kit below
(annotations, `outputSchema`, in-band error wrapping, per-field `.describe()`)
would not apply to `memory_ingest` / `memory_ingest_status`, contradicting the
"all 11 tools" acceptance criteria.
Therefore, as part of the polish-kit PR (PR 2), one of the following must
happen — the implementation plan picks:
1. **Preferred:** Move `memory_ingest` and `memory_ingest_status` registration
into `registerKtxContextTools` so they go through `registerParsedTool` like
every other tool. The `MemoryIngestPort` becomes a `contextTools.memoryIngest`
port and the standalone `registerMemoryCaptureTools` helper in `server.ts`
is deleted.
2. **Acceptable fallback:** Keep `registerMemoryIngestTools` in `server.ts`
but rewrite it to call `registerParsedTool` (exported from
`context-tools.ts`) so the same annotations / `outputSchema` /
error-wrapping plumbing is applied uniformly.
Either way, every checklist item in §§2.1–2.4 must apply to the two memory
tools, and the §Verification annotations and `outputSchema` tests must cover
them.
#### 2.1 Tool annotations
Every tool gets annotations and a `title`:
| Tool | title | readOnly | destructive | idempotent | openWorld |
|---|---|:--:|:--:|:--:|:--:|
| `connection_list` | Connection List | ✓ | — | ✓ | — |
| `discover_data` | Discover Data | ✓ | — | — | — |
| `wiki_search` | Wiki Search | ✓ | — | — | — |
| `wiki_read` | Wiki Read | ✓ | — | ✓ | — |
| `entity_details` | Entity Details | ✓ | — | ✓ | — |
| `dictionary_search` | Dictionary Search | ✓ | — | — | — |
| `sl_read_source` | Semantic Layer Read Source | ✓ | — | ✓ | — |
| `sl_query` | Semantic Layer Query | ✓ | — | — | — |
| `sql_execution` | SQL Execution | ✓ | — | — | — |
| `memory_ingest` | Memory Ingest | — | ✓ | — | — |
| `memory_ingest_status` | Memory Ingest Status | ✓ | — | omit | — |
`openWorldHint: false` for every tool — even `sql_execution` targets a
configured, bounded warehouse, not the web. `sql_execution` is `readOnlyHint:
true` because the server-side parser enforces read-only (`assertReadOnlySql`).
`destructiveHint` is omitted (defaults to `false`) for read-only tools per the
MCP spec; explicit `false` is fine but redundant.
`ToolAnnotations` are static optional booleans per the MCP 2025-11-25 schema
(`title?`, `readOnlyHint?`, `destructiveHint?`, `idempotentHint?`,
`openWorldHint?` — no state-dependent variants). `idempotentHint` describes
whether repeated calls have additional environmental effect and is most
meaningful when `readOnlyHint` is `false`. For `memory_ingest_status`, which is
a polling read whose response shape changes while a run is active, leave
`idempotentHint` unset — the tool is read-only but not statically idempotent.
`registerTool` accepts annotations in the `config` object today; this is a
plumbing change in `registerParsedTool` to forward them.
#### 2.2 `outputSchema` on all 11 tools
Per the MCP 2025-11-25 spec, clients SHOULD validate `structuredContent`
against `outputSchema` when declared. Authoring is mechanical: each response
shape already typed in `packages/context/src/mcp/types.ts` gets a parallel
Zod schema and is passed as `outputSchema` to `registerTool`.
`registerParsedTool` is extended to accept an optional `outputSchema` arg and
forward it to `server.registerTool`. The Zod schemas live alongside the
input schemas in `context-tools.ts` (or a sibling `tool-output-schemas.ts` if
the file grows too large).
Example for `discover_data`:
```typescript
const discoverDataOutputSchema = z.object({
refs: z.array(
z.object({
kind: discoverDataKindSchema,
id: z.string(),
score: z.number(),
summary: z.string().nullable(),
snippet: z.string().nullable(),
matchedOn: z.enum(['name', 'display', 'description', 'comment', 'expr', 'sample_value', 'body']),
connectionId: z.string().optional(),
tableRef: z.object({ catalog: z.string().nullable(), db: z.string().nullable(), name: z.string() }).optional(),
columnName: z.string().optional(),
}),
),
});
```
#### 2.3 Per-field `.describe()` on every input
Anthropic's documented mechanism for fighting model drift, already used in
`memory_capture*`. Applied to every input field on every retained tool.
Highest leverage: `sl_query`, `entity_details`, `dictionary_search`,
`sql_execution`, `memory_ingest`. Tool-level `description` strings are
rewritten to be longer with one concrete example shape inlined (the technique
that fixed `order_by` model drift in this session).
#### 2.4 In-band error wrapping in `registerParsedTool`
Per MCP spec, tools return handler/runtime errors as `isError: true` + text
content, not JSON-RPC errors. Move the try/catch into the `registerParsedTool`
helper so every tool consistently surfaces handler exceptions as
`jsonErrorToolResult`. `sql_execution`'s local try/catch is removed (the
helper handles it).
**Scope — what becomes in-band vs. what stays JSON-RPC.** The MCP SDK
pre-validates incoming arguments against the registered `inputSchema` before
the tool callback runs, and surfaces validation failures as
`McpError(InvalidParams)` / JSON-RPC errors
(`@modelcontextprotocol/sdk/dist/esm/server/mcp.js` `validateToolInput`,
~line 166). KTX cannot intercept those without forking the SDK and we will not.
Therefore:
- Schema-validation failures on input → remain JSON-RPC `InvalidParams` errors,
emitted by the SDK before our handler runs. This is the documented MCP
behavior; clients already handle it.
- Handler exceptions, port/driver errors, and any post-validation runtime
errors thrown inside the tool body → wrapped in-band as
`{ isError: true, content: [{ type: 'text', ... }] }` by
`registerParsedTool`'s catch.
- The redundant `inputSchema.parse(input)` inside `registerParsedTool` may be
kept as defense-in-depth (e.g., for the rare path where the SDK was given a
raw shape and a downstream change loosens validation) or removed; either is
acceptable. If kept, parse failures here are wrapped in-band as well, but in
practice they are unreachable for valid SDK registrations because the SDK
has already parsed against the same schema.
```typescript
function registerParsedTool<TInput extends z.ZodType, TOutput extends z.ZodType>(
server: KtxMcpServerLike,
name: string,
config: { title: string; description: string; inputSchema: unknown; outputSchema?: unknown; annotations: ToolAnnotations },
inputSchema: TInput,
handler: (input: z.infer<TInput>) => Promise<KtxMcpToolResult>,
outputSchema?: TOutput,
): void {
server.registerTool(name, { ...config, outputSchema: outputSchema ? outputSchema : undefined }, async (input) => {
try {
return await handler(inputSchema.parse(input));
} catch (error) {
return jsonErrorToolResult(formatToolError(error));
}
});
}
```
A small `formatToolError` helper renders Zod errors with `path: message` lines
and falls through to `error.message` / `String(error)` for non-Zod cases.
Acceptance tests in §Verification must therefore split the error path:
- Bad input shape (rejected by SDK pre-validation) → expect a thrown
`McpError`/`InvalidParams` JSON-RPC error, not `isError: true`.
- Handler-thrown / port-thrown error (e.g., unknown `connectionId`, driver
failure) → expect `{ isError: true, content: [{ type: 'text', ... }] }`.
#### 2.5 Union-drift normalization
Apply the same `z.preprocess` pattern used for `order_by` to the two remaining
at-risk unions:
**`slQueryDimensionSchema`** — accept `{ dimension, granularity }` (Cube
convention) as an alias for `{ field, granularity }`. Bare strings continue to
work unchanged.
```typescript
const slQueryDimensionSchema = z.preprocess(
(value) => {
if (typeof value === 'string') return { field: value };
if (value && typeof value === 'object' && !Array.isArray(value)) {
const obj = { ...(value as Record<string, unknown>) };
if (!('field' in obj) && typeof obj.dimension === 'string') obj.field = obj.dimension;
return obj;
}
return value;
},
z.object({
field: z.string().min(1).describe('Dimension to group by, e.g. "orders.created_at" or a SL dimension key.'),
granularity: z.string().min(1).optional().describe('Time granularity for time dimensions: day, week, month, quarter, year.'),
}),
);
```
**`entityDetailsTableRefSchema`** — accept `{ schema, table }` (BigQuery /
SQL-style convention) as an alias for `{ db, name }`. Today's schema requires
`catalog`, `db`, and `name` (`packages/context/src/mcp/context-tools.ts:169`),
so the alias path must also default `catalog` to `null` to satisfy the
validator. Either of the two equivalent shapes below is acceptable; the
implementation plan picks one:
1. Make `catalog` (and `db`) `.nullable().default(null)` so the alias path
doesn't have to set them, and bare `{ schema, table }` is accepted with
`catalog === null`, `db === schema`, `name === table`.
2. Have the preprocess unconditionally fill missing `catalog`/`db` with `null`.
Acceptance criterion: a tool call with `{ table: { schema: "public", table: "orders" } }` parses successfully and resolves to `{ catalog: null, db: "public", name: "orders" }`. Existing callers passing `{ catalog, db, name }` continue to work unchanged.
```typescript
const entityDetailsTableRefSchema = z.preprocess(
(value) => {
if (value && typeof value === 'object' && !Array.isArray(value)) {
const obj = { ...(value as Record<string, unknown>) };
if (!('db' in obj) && typeof obj.schema === 'string') obj.db = obj.schema;
if (!('name' in obj) && typeof obj.table === 'string') obj.name = obj.table;
if (!('catalog' in obj)) obj.catalog = null;
return obj;
}
return value;
},
z.object({
catalog: z.string().nullable().describe('Catalog/project (BigQuery project, Snowflake database). null when not applicable.'),
db: z.string().nullable().describe('Schema/database for the table. null when not applicable.'),
name: z.string().min(1).describe('Table name.'),
}),
);
```
`slQueryMeasureSchema` is **not** changed — bare strings cover the common case
and `{ expr, name }` matches Cube's measure-with-alias convention; no observed
drift.
#### 2.6 Type-narrow `jsonToolResult`
The goal is to forbid arrays at compile time without breaking the existing
`interface`-typed response shapes (e.g. `KtxKnowledgeSearchResponse` in
`packages/context/src/mcp/types.ts`). `Record<string, unknown>` is **not**
acceptable as the constraint because TypeScript interfaces without an index
signature are not assignable to it; using it would cause widespread compile
failures on valid object responses.
Use a non-array object constraint instead, e.g.:
```typescript
type NonArrayObject = object & { length?: never };
export function jsonToolResult<T extends NonArrayObject>(
structuredContent: T,
): KtxMcpToolResult<T> { ... }
```
Acceptance criteria:
- A bare array literal at a call site fails to type-check (catches the
`discover_data` bug class).
- Every existing `interface`-typed response (`KtxKnowledgeSearchResponse`,
`KtxSemanticLayerQueryResponse`, `KtxSqlExecutionResponse`,
`KtxEntityDetailsResponse`, `KtxDictionarySearchResponse`,
`KtxDiscoverDataResponse`, `KtxConnectionTestResponse`,
`KtxSemanticLayerReadResponse`, `KtxSemanticLayerListResponse`,
`KtxKnowledgePage`, memory capture/status response shapes) continues to
type-check at every existing `jsonToolResult` call site without modification.
Implementation plan can substitute an equivalent narrowing if it is more
idiomatic; the contract is "no arrays, no breaking interface assignability."
Pure defensive type change; no runtime effect on current code.
#### 2.7 Enforce the `toResolvedWire` invariant
Add a doc comment on `KtxSemanticLayerComputePort.query` and
`.validateSources` stating that callers must pass `toResolvedWire`-sanitized
sources to prevent the daemon `usage`-leak bug from regressing if a new code
path bypasses `SemanticLayerService`.
The doc comment alone is not sufficient — there is already an unsanitized
caller. `loadComputableSources` in
`packages/context/src/mcp/local-project-ports.ts` (~line 311) parses YAML and
pushes the raw record into `validateSources`
(`packages/context/src/mcp/local-project-ports.ts:586`). It must be brought into
conformance by sanitizing each record with `toResolvedWire` before handing it
to `validateSources`, mirroring the existing sanitization in
`packages/context/src/sl/local-query.ts:76`. Acceptance criterion: every
`KtxSemanticLayerComputePort.query` and `.validateSources` call site in the
repo passes `toResolvedWire`-sanitized records, verified by code review and
covered by the existing `local-query.test.ts` / `local-project-ports.test.ts`
tests for the relevant paths. Note that `sl_validate` is removed from MCP
registration (§1.2) but the underlying port keeps backing the CLI, so the
invariant must hold for the CLI path too.
#### 2.8 Progress notifications — `sql_execution` + `sl_query`
Per MCP spec, the caller may include `params._meta.progressToken` in the
request; the server emits `notifications/progress` with `{ progressToken,
progress, total?, message }` at stage transitions.
`KtxSqlExecutionMcpPort.execute` and `KtxSemanticLayerMcpPort.query` are
extended with an optional `onProgress?: (event: { progress: number; total?: number; message: string }) => void` parameter. The MCP tool handlers wire
`onProgress` to the SDK's notification channel via the handler-context object
passed by `server.registerTool`. Non-progress-supporting clients ignore the
events.
Emitted stages:
- `sql_execution`: `"Validating SQL"` (progress 0.0) → `"Executing"` (0.3) → `"Fetched N rows"` (1.0).
- `sl_query`: `"Compiling query"` (0.0) → `"Generating SQL"` (0.3) → `"Executing"` (0.6) → `"Fetched N rows"` (1.0).
Progress emission is best-effort; if the underlying port can't report a stage
boundary (e.g., a driver doesn't expose progress callbacks), the stage is
simply skipped.
`memory_ingest` does not emit progress — `runId` + `memory_ingest_status`
polling is the documented async pattern for it.
### 3. `ktx-analytics` SKILL.md refinements
File: `packages/cli/src/skills/analytics/SKILL.md`.
**Step 7 rewrite** (current vs. new):
Current:
> 7. **Capture durable learnings** - at the end of the turn, call `memory_capture` when the investigation produced reusable business context, metric definitions, or schema knowledge.
New:
> 7. **Capture durable learnings** - call `memory_ingest` whenever a turn produces something worth remembering (business rules, metric definitions, schema gotchas, recurring findings) **or** whenever the user asks you to remember something. Pass markdown in `content` including any source context the memory agent should weigh. Each call is a feedback loop — better notes today mean smarter `discover_data` and `wiki_search` results tomorrow.
**Tool-name updates** throughout: every `memory_capture` reference becomes
`memory_ingest`.
**Multi-connection rule** added under `<rules>` — phrased to match the §1.1
connection matrix so the agent does not over-scope unscoped tools:
> When `connection_list` shows multiple connections, pass an explicit
> `connectionId` to every tool that takes one **and where user intent pins a
> specific warehouse**. The matrix is:
>
> - **Required:** `entity_details`, `sl_read_source`, `sql_execution`.
> - **Required when user intent is warehouse-specific (including any wording
> like "in our warehouse" / "this warehouse"):** `memory_ingest` — without
> `connectionId`, the memory agent cannot update the semantic layer and the
> knowledge will land as wiki-only.
> - **Pass when intent pins a warehouse, otherwise omit for unscoped
> discovery:** `sl_query`, `discover_data`, `dictionary_search`.
> - **Never pass `connectionId` (the tool does not accept one):**
> `connection_list`, `wiki_search`, `wiki_read`, `memory_ingest_status`.
>
> If intent is ambiguous for a required-or-scoped tool, ask the user which
> warehouse before calling — do not guess.
**One new worked example** demonstrating user-driven ingest:
> **Input:** "Heads up — ARR is always reported in cents in our warehouse."
>
> **Workflow:**
> 1. If multiple connections, call `connection_list` and pick the warehouse the
> user means (asking if ambiguous). Pass its id as `connectionId` so the
> memory agent can update the semantic layer, not just the wiki.
> 2. `memory_ingest({ connectionId: "<warehouse-id>", content: "ARR is reported in cents (not dollars) in this warehouse. Multiply by 0.01 for dollar amounts. Source: user clarification." })` — no analysis turn; just remember.
The existing Discover → Inspect → Resolve → Plan → Query → Validate → Capture
workflow stays. The existing two examples are updated only to reflect the
`memory_capture` → `memory_ingest` rename.
## Migration / sequencing
Three landings, each independently mergeable, in this order:
### PR 1 — Surface change (atomic)
- Remove the 14 admin tools from `registerKtxContextTools` (conditional
registration blocks deleted).
- Rename `memory_capture` → `memory_ingest`, `memory_capture_status` →
`memory_ingest_status`. Rename `MemoryCapturePort` → `MemoryIngestPort`,
`MemoryCaptureService` → `MemoryIngestService`. Update the new tool's input
contract per §1.3.
- Update `packages/context/src/mcp/local-project-ports.ts` and
`packages/context/src/mcp/server.ts` to reflect the renames and the dropped
ports.
- Update `packages/cli/src/skills/analytics/SKILL.md` per §3 in the same
diff.
- Update all tests for the removed/renamed tools.
- Update `docs-site/content/docs/integrations/agent-clients.mdx` to replace
the existing "memory capture" wording (currently at line ~90) with
"memory ingest". This update is unconditional — the file already names the
tool family.
- Update `packages/cli/src/mcp-server-factory.ts` and
`packages/cli/src/text-ingest.ts` (plus their tests) to reflect the
`MemoryCapture*` → `MemoryIngest*` rename. Both files import
`createLocalProjectMemoryCapture` and use a `memoryCapture` variable, so the
rename does cross the CLI boundary even though the tool surface used by the
CLI is unchanged. Re-export rename in `packages/context/src/memory/index.ts`
is part of this PR.
The CLI's runtime use of the removed admin-tool implementations is unchanged —
only the memory rename touches CLI code.
### PR 2 — Polish kit
Touches all 11 retained tools. Can be one PR or split per family
(annotations + outputSchema + descriptions + error wrapping + union-drift
fixes + `jsonToolResult` type narrowing + `toResolvedWire` doc comment).
### PR 3 — Progress notifications
Extends `KtxSqlExecutionMcpPort` and `KtxSemanticLayerMcpPort` with optional
`onProgress`, wires the MCP handler context's notification channel, emits at
the stage boundaries listed in §2.8.
Eventual **deletion** of the 14 admin tool implementations is a separate
follow-up spec gated on the `ktx-admin` SKILL landing. Until then they remain
in `packages/context/src/` and are used only by the CLI.
## Verification
### Unit tests per retained tool
For each of the 11 retained tools:
- Input schema accepts canonical input.
- Input schema accepts each documented normalized alt-shape (Cube
`{ dimension, granularity }`, `{ schema, table }`, bare-string `order_by`,
Cube-style `{ id, desc }` `order_by`).
- Output schema accepts the response shape returned by the underlying port.
- Error path returns `{ isError: true, content: [{ type: 'text', ... }] }`
(not a thrown exception).
### Schema snapshot test
A `tools/list` snapshot test in `packages/context/src/mcp/server.test.ts`
captures the exact JSON Schema each client receives for every tool. Re-runs
across PRs catch accidental schema drift (e.g., a Zod change silently
broadening the contract).
### Annotations test
Assert every tool's registered config carries the expected `readOnlyHint`,
`destructiveHint`, `idempotentHint`, `openWorldHint`, and `title` per §2.1.
### Multi-client end-to-end smoke
Stdio (Claude Desktop) and Streamable HTTP (Claude Code) are the two
transports; the other four clients (Codex, Cursor, OpenCode, universal) share
one of these transports and their config files are static. Spin up `ktx mcp
stdio` and `ktx mcp start`, call each retained tool through both transports,
verify response shape against `outputSchema`.
### Required commands
The named test files include slow tests that are excluded from the default
`@ktx/context` `test` script and live in `test:slow`
(`packages/context/package.json:127-128`). Implementation must run both:
```bash
pnpm --filter @ktx/context run test
pnpm --filter @ktx/context run test:slow
pnpm --filter @ktx/context run type-check
pnpm --filter @ktx/cli run type-check
pnpm --filter @ktx/cli run test
pnpm run dead-code
```
CLI checks are required because PR 1 renames cross the CLI boundary
(`packages/cli/src/mcp-server-factory.ts`, `packages/cli/src/text-ingest.ts`,
the `MemoryCapture*` re-exports, plus their vitest specs).
`pnpm --filter @ktx/cli run test:slow` should also be added if PR 1 ends up
touching any of the slow-test files enumerated in `packages/cli/package.json`
(`scan.test.ts`, `setup*.test.ts`, etc.); the rename diff today does not, but
the implementation plan must re-check before merging.
When `docs-site/content/docs/integrations/agent-clients.mdx` is touched, also
run the docs-site scripts declared in `docs-site/package.json` — there is no
`lint` script:
```bash
pnpm --filter ktx-docs run build
pnpm --filter ktx-docs run test
```
### Red-green regression
For the union-drift fixes (§2.5): revert the preprocess in `local-query.ts` or
`context-tools.ts`, run the alt-shape test → expect failure, restore →
expect pass. Same pattern as the `order_by` and `usage`-leak fixes in this
session.
## Risks
- **Removed tools surprise a user who depended on them via MCP.** Mitigated
by the no-back-compat rule (KTX is pre-public) and by the SKILL update
landing atomically with the surface change. Users on `ktx setup --agents`
flow get the updated SKILL the next time they re-run setup.
- **`outputSchema` validation breaks a client that doesn't tolerate
unrecognized JSON Schema keywords.** Mitigated by emitting `outputSchema`
via the same `server.registerTool` path that already produces `inputSchema`,
so both schemas are serialized by the MCP SDK as JSON Schema 2020-12 (the
dialect the SDK's tool-list types declare —
`@modelcontextprotocol/sdk/.../types.js` `inputSchema` / `outputSchema`
fields, and Appendix B). The spec uses SHOULD-validate semantics, not MUST.
The snapshot test catches drift; the multi-client smoke confirms
compatibility.
- **Progress notifications increase notification volume for clients that
poll.** Mitigated by stage-based emission (3-4 events per call max). Clients
that don't support progress simply ignore the events.
- **Renaming `MemoryCapturePort`/`MemoryCaptureService` cascades through
internal callers.** Cascade is bounded to `packages/context/src/memory/`,
`packages/context/src/mcp/`, and their tests; type-checker catches missed
call sites.
## Open Questions
None at design time. Open items for the implementation plan:
- Final wording for the rewritten tool descriptions on `discover_data`,
`entity_details`, `dictionary_search`, `sl_read_source`, `sl_query`,
`sql_execution`. (Drafts can be authored during PR 2.)
- Whether `formatToolError` should redact path elements for security
(probably not — these are local-only MCP servers, and the existing error
shape doesn't redact).
- Whether to split PR 2 into per-family sub-PRs or keep it monolithic. Default
is monolithic since the polish-kit changes touch the same files and tests.
## Appendix A — File map
| Change | File |
|---|---|
| Tool registration removed (14 tools) | `packages/context/src/mcp/context-tools.ts` |
| Memory rename + route memory_ingest tools through the shared polish path (§2.0) | `packages/context/src/mcp/server.ts`, `packages/context/src/mcp/context-tools.ts`, `packages/context/src/memory/*` |
| Local ports update | `packages/context/src/mcp/local-project-ports.ts` |
| Port types update | `packages/context/src/mcp/types.ts` |
| Annotations, outputSchema, describe, error wrapping, union preprocess | `packages/context/src/mcp/context-tools.ts` |
| `jsonToolResult` type narrowing | `packages/context/src/mcp/context-tools.ts` |
| `toResolvedWire` invariant comment | `packages/context/src/daemon/semantic-layer-compute.ts` |
| Progress callback plumbing | `packages/context/src/daemon/semantic-layer-compute.ts`, `packages/context/src/sl/local-query.ts`, `packages/context/src/mcp/local-project-ports.ts` (`executeValidatedReadOnlySql`) |
| Tests | `packages/context/src/mcp/server.test.ts`, `packages/context/src/mcp/local-project-ports.test.ts`, `packages/context/src/sl/local-query.test.ts` |
| Skill | `packages/cli/src/skills/analytics/SKILL.md` |
| Docs | `docs-site/content/docs/integrations/agent-clients.mdx` |
## Appendix B — MCP-spec cross-reference
- Tool annotations (`readOnlyHint`, `destructiveHint`, `idempotentHint`,
`openWorldHint`, `title`): MCP 2025-11-25 spec `/schema` "ToolAnnotations".
- `outputSchema` with `structuredContent` SHOULD-validate: spec
`/server/tools` "Tool Result > Structured Content".
- In-band error contract (`isError: true` + text content, not JSON-RPC
error): spec `/server/tools` "Tool Execution Error Example".
- Progress notifications via `params._meta.progressToken` →
`notifications/progress`: spec `/basic/utilities/progress`.
- `inputSchema.type: "object"` requirement and JSON Schema 2020-12 default
dialect: spec `/server/tools` "Tool > inputSchema" and SEP 1613.

1
packages/cli/package.json
View file

@ -48,6 +48,7 @@
"@ktx/llm": "workspace:*",
"@modelcontextprotocol/sdk": "^1.29.0",
"commander": "14.0.3",
"fflate": "^0.8.2",
"ink": "^7.0.2",
"react": "^19.2.6",
"zod": "^4.4.3"

4
packages/cli/src/cli-program.ts
View file

@ -186,6 +186,10 @@ function shouldSuppressProjectDirLine(path: string[], options: Record<string, un
return true;
}
if (commandPathKey === 'ktx mcp stdio') {
return true;
}
if (
commandPathKey === 'ktx status' &&
typeof options.projectDir !== 'string' &&

1
packages/cli/src/cli-runtime.ts
View file

@ -39,6 +39,7 @@ export interface KtxCliDeps {
stopDaemon?: typeof import('./managed-mcp-daemon.js').stopKtxMcpDaemon;
readStatus?: typeof import('./managed-mcp-daemon.js').readKtxMcpDaemonStatus;
runServer?: typeof import('./mcp-http-server.js').runKtxMcpHttpServer;
runStdioServer?: typeof import('./mcp-stdio-server.js').runKtxMcpStdioServer;
};
}

45
packages/cli/src/commands/mcp-commands.test.ts
View file

@ -35,6 +35,7 @@ describe('registerMcpCommands', () => {
'serve-internal',
'start',
'status',
'stdio',
'stop',
]);
expect(
@ -54,4 +55,48 @@ describe('registerMcpCommands', () => {
);
expect(startDaemon).not.toHaveBeenCalled();
});
it('prints "already running" when startDaemon reports already-running', async () => {
const program = new Command().exitOverride().option('--project-dir <path>');
const startDaemon = vi.fn().mockResolvedValue({
status: 'already-running',
url: 'http://127.0.0.1:7878/mcp',
state: {
schemaVersion: 1,
pid: 4242,
host: '127.0.0.1',
port: 7878,
tokenAuth: false,
projectDir: '/tmp/ktx-already',
startedAt: '2026-05-14T00:00:00.000Z',
logPath: '/tmp/ktx-already/.ktx/logs/mcp.log',
},
});
const context = makeContext({ deps: { mcp: { startDaemon } } });
registerMcpCommands(program, context);
await program.parseAsync(['--project-dir', '/tmp/ktx-already', 'mcp', 'start'], { from: 'user' });
expect(startDaemon).toHaveBeenCalledTimes(1);
expect(context.io.stdout.write).toHaveBeenCalledWith(
'KTX MCP daemon already running: http://127.0.0.1:7878/mcp\n',
);
});
it('runs the stdio server with the resolved project directory', async () => {
const program = new Command().exitOverride().option('--project-dir <path>');
const runStdioServer = vi.fn().mockResolvedValue(undefined);
const context = makeContext({ deps: { mcp: { runStdioServer } } });
registerMcpCommands(program, context);
await expect(program.parseAsync(['--project-dir', '/tmp/ktx6', 'mcp', 'stdio'], { from: 'user' })).resolves.toBe(
program,
);
expect(runStdioServer).toHaveBeenCalledWith({
projectDir: '/tmp/ktx6',
cliVersion: '0.0.0-test',
io: context.io,
});
});
});

18
packages/cli/src/commands/mcp-commands.ts
View file

@ -15,6 +15,7 @@ import {
stopKtxMcpDaemon,
} from '../managed-mcp-daemon.js';
import { buildMcpSecurityConfig, runKtxMcpHttpServer } from '../mcp-http-server.js';
import { runKtxMcpStdioServer } from '../mcp-stdio-server.js';
function tokenFromOption(value: string | undefined): string | undefined {
return value ?? process.env.KTX_MCP_TOKEN;
@ -27,6 +28,17 @@ function binPath(): string {
export function registerMcpCommands(program: Command, context: KtxCliCommandContext): void {
const mcp = program.command('mcp').description('Run the KTX MCP HTTP server');
mcp
.command('stdio')
.description('Run the KTX MCP server over stdio')
.action(async (_options, command) => {
await (context.deps.mcp?.runStdioServer ?? runKtxMcpStdioServer)({
projectDir: resolveCommandProjectDir(command),
cliVersion: context.packageInfo.version,
io: context.io,
});
});
mcp
.command('start')
.description('Start the KTX MCP HTTP server')
@ -70,7 +82,11 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
allowedOrigins: options.allowedOrigin,
binPath: binPath(),
});
context.io.stdout.write(`KTX MCP daemon started: ${result.url}\n`);
context.io.stdout.write(
result.status === 'started'
? `KTX MCP daemon started: ${result.url}\n`
: `KTX MCP daemon already running: ${result.url}\n`,
);
});
mcp

1
packages/cli/src/commands/setup-commands.ts
View file

@ -218,6 +218,7 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
.addOption(
new Option('--target <target>', 'Agent target').choices([
'claude-code',
'claude-desktop',
'codex',
'cursor',
'opencode',

72
packages/cli/src/managed-mcp-daemon.test.ts
View file

@ -94,6 +94,78 @@ describe('managed MCP daemon lifecycle', () => {
);
});
it('returns already-running without spawning when the daemon is alive at the same host/port', async () => {
await mkdir(join(projectDir, '.ktx'), { recursive: true });
await writeFile(join(projectDir, '.ktx/mcp.json'), `${JSON.stringify(state(projectDir), null, 2)}\n`);
const spawnDaemon = vi.fn(() => child(9999));
const result = await startKtxMcpDaemon({
projectDir,
cliVersion: '0.0.0-test',
host: '127.0.0.1',
port: 7878,
allowedHosts: [],
allowedOrigins: [],
binPath: '/repo/packages/cli/dist/bin.js',
spawnDaemon,
processAlive: vi.fn(() => true),
portAvailable: vi.fn(async () => true),
});
expect(result.status).toBe('already-running');
expect(result.url).toBe('http://127.0.0.1:7878/mcp');
expect(result.state.pid).toBe(4242);
expect(spawnDaemon).not.toHaveBeenCalled();
});
it('throws when the recorded daemon uses a different host or port', async () => {
await mkdir(join(projectDir, '.ktx'), { recursive: true });
await writeFile(join(projectDir, '.ktx/mcp.json'), `${JSON.stringify(state(projectDir), null, 2)}\n`);
const spawnDaemon = vi.fn(() => child(9999));
await expect(
startKtxMcpDaemon({
projectDir,
cliVersion: '0.0.0-test',
host: '127.0.0.1',
port: 9000,
allowedHosts: [],
allowedOrigins: [],
binPath: '/repo/packages/cli/dist/bin.js',
spawnDaemon,
processAlive: vi.fn(() => true),
portAvailable: vi.fn(async () => true),
}),
).rejects.toThrow(/different configuration[\s\S]*ktx mcp stop/);
expect(spawnDaemon).not.toHaveBeenCalled();
});
it('throws when token-auth presence differs from the recorded daemon', async () => {
await mkdir(join(projectDir, '.ktx'), { recursive: true });
await writeFile(
join(projectDir, '.ktx/mcp.json'),
`${JSON.stringify(state(projectDir, { tokenAuth: false }), null, 2)}\n`,
);
const spawnDaemon = vi.fn(() => child(9999));
await expect(
startKtxMcpDaemon({
projectDir,
cliVersion: '0.0.0-test',
host: '127.0.0.1',
port: 7878,
token: 'secret-token',
allowedHosts: [],
allowedOrigins: [],
binPath: '/repo/packages/cli/dist/bin.js',
spawnDaemon,
processAlive: vi.fn(() => true),
portAvailable: vi.fn(async () => true),
}),
).rejects.toThrow(/different configuration/);
expect(spawnDaemon).not.toHaveBeenCalled();
});
it('reports running when the process is alive and health passes', async () => {
await mkdir(join(projectDir, '.ktx'), { recursive: true });
await writeFile(join(projectDir, '.ktx/mcp.json'), `${JSON.stringify(state(projectDir), null, 2)}\n`);

18
packages/cli/src/managed-mcp-daemon.ts
View file

@ -121,11 +121,25 @@ export async function startKtxMcpDaemon(options: {
portAvailable?: (host: string, port: number) => Promise<boolean>;
spawnDaemon?: typeof defaultSpawnDaemon;
now?: () => Date;
}): Promise<{ status: 'started'; state: KtxMcpDaemonState; url: string }> {
}): Promise<{ status: 'started' | 'already-running'; state: KtxMcpDaemonState; url: string }> {
const existing = await readState(options.projectDir).catch(() => undefined);
const processAlive = options.processAlive ?? defaultProcessAlive;
if (existing && processAlive(existing.pid)) {
throw new Error(`KTX MCP daemon is already recorded at http://${existing.host}:${existing.port}/mcp`);
const sameConfig =
existing.host === options.host &&
existing.port === options.port &&
existing.tokenAuth === Boolean(options.token);
if (sameConfig) {
return {
status: 'already-running',
state: existing,
url: `http://${existing.host}:${existing.port}/mcp`,
};
}
throw new Error(
`KTX MCP daemon is already running at http://${existing.host}:${existing.port}/mcp ` +
'with a different configuration. Run `ktx mcp stop` first, then start again.',
);
}
const portAvailable = options.portAvailable ?? defaultPortAvailable;
if (!(await portAvailable(options.host, options.port))) {

67
packages/cli/src/mcp-http-server.ts
View file

@ -1,16 +1,11 @@
import { randomUUID } from 'node:crypto';
import { createServer, type IncomingHttpHeaders, type IncomingMessage, type Server, type ServerResponse } from 'node:http';
import { createDefaultKtxMcpServer, createLocalProjectMcpContextPorts } from '@ktx/context/mcp';
import { createLocalProjectMemoryCapture } from '@ktx/context/memory';
import { loadKtxProject, type KtxLocalProject } from '@ktx/context/project';
import { loadKtxProject } from '@ktx/context/project';
import type { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import { isInitializeRequest } from '@modelcontextprotocol/sdk/types.js';
import type { KtxCliIo } from './cli-runtime.js';
import { createKtxCliIngestQueryExecutor } from './ingest-query-executor.js';
import { createKtxCliScanConnector } from './local-scan-connectors.js';
import { createManagedPythonSemanticLayerComputePort } from './managed-python-command.js';
import { createManagedDaemonSqlAnalysisPort } from './managed-python-http.js';
import { createKtxMcpServerFactory } from './mcp-server-factory.js';
const DEFAULT_ALLOWED_HOSTS = ['localhost', '127.0.0.1', '::1'] as const;
@ -124,13 +119,6 @@ export interface RunKtxMcpHttpServerOptions extends McpSecurityConfigInput {
loadProject?: typeof loadKtxProject;
}
function noopIo(): KtxCliIo {
return {
stdout: { write() {} },
stderr: { write() {} },
};
}
function writeJson(res: ServerResponse, status: number, body: object): void {
const payload = `${JSON.stringify(body)}\n`;
res.writeHead(status, {
@ -159,55 +147,6 @@ async function readJsonBody(req: IncomingMessage): Promise<unknown> {
return raw.trim().length === 0 ? undefined : (JSON.parse(raw) as unknown);
}
async function defaultMcpServerFactory(input: {
project: KtxLocalProject;
projectDir: string;
cliVersion: string;
io?: KtxCliIo;
}): Promise<() => McpServer> {
const io = input.io ?? noopIo();
const queryExecutor = createKtxCliIngestQueryExecutor(input.project);
const semanticLayerCompute = await createManagedPythonSemanticLayerComputePort({
cliVersion: input.cliVersion,
installPolicy: 'auto',
io,
});
const sqlAnalysis = createManagedDaemonSqlAnalysisPort({
cliVersion: input.cliVersion,
projectDir: input.projectDir,
installPolicy: 'auto',
io,
});
const contextTools = createLocalProjectMcpContextPorts(input.project, {
semanticLayerCompute,
queryExecutor,
sqlAnalysis,
localScan: {
createConnector: async (connectionId) => createKtxCliScanConnector(input.project, connectionId),
},
localIngest: {
semanticLayerCompute,
queryExecutor,
},
});
let memoryCapture: ReturnType<typeof createLocalProjectMemoryCapture> | undefined;
try {
memoryCapture = createLocalProjectMemoryCapture(input.project, { semanticLayerCompute, queryExecutor });
} catch (error) {
input.io?.stderr.write(`KTX MCP memory_capture disabled: ${error instanceof Error ? error.message : String(error)}\n`);
}
return () =>
createDefaultKtxMcpServer({
name: 'ktx',
version: input.cliVersion,
userContext: { userId: 'local' },
contextTools,
memoryCapture,
});
}
function listenerPort(server: Server, fallback: number): number {
const address = server.address();
return typeof address === 'object' && address ? address.port : fallback;
@ -233,7 +172,7 @@ export async function runKtxMcpHttpServer(options: RunKtxMcpHttpServerOptions):
: undefined;
const createMcpServer =
options.createMcpServer ??
(await defaultMcpServerFactory({
(await createKtxMcpServerFactory({
project: project!,
projectDir: options.projectDir,
cliVersion: options.cliVersion ?? '0.0.0-private',

63
packages/cli/src/mcp-server-factory.ts Normal file
View file

@ -0,0 +1,63 @@
import { createDefaultKtxMcpServer, createLocalProjectMcpContextPorts } from '@ktx/context/mcp';
import { createLocalProjectMemoryIngest } from '@ktx/context/memory';
import type { KtxLocalProject } from '@ktx/context/project';
import type { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import type { KtxCliIo } from './cli-runtime.js';
import { createKtxCliIngestQueryExecutor } from './ingest-query-executor.js';
import { createKtxCliScanConnector } from './local-scan-connectors.js';
import { createManagedPythonSemanticLayerComputePort } from './managed-python-command.js';
import { createManagedDaemonSqlAnalysisPort } from './managed-python-http.js';
function noopMcpIo(): KtxCliIo {
return {
stdout: { write() {} },
stderr: { write() {} },
};
}
export async function createKtxMcpServerFactory(input: {
project: KtxLocalProject;
projectDir: string;
cliVersion: string;
io?: KtxCliIo;
}): Promise<() => McpServer> {
const io = input.io ?? noopMcpIo();
const queryExecutor = createKtxCliIngestQueryExecutor(input.project);
const semanticLayerCompute = await createManagedPythonSemanticLayerComputePort({
cliVersion: input.cliVersion,
installPolicy: 'auto',
io,
});
const sqlAnalysis = createManagedDaemonSqlAnalysisPort({
cliVersion: input.cliVersion,
projectDir: input.projectDir,
installPolicy: 'auto',
io,
});
const contextTools = createLocalProjectMcpContextPorts(input.project, {
semanticLayerCompute,
queryExecutor,
sqlAnalysis,
localScan: {
createConnector: async (connectionId) => createKtxCliScanConnector(input.project, connectionId),
},
});
let memoryIngest: ReturnType<typeof createLocalProjectMemoryIngest> | undefined;
try {
memoryIngest = createLocalProjectMemoryIngest(input.project, { semanticLayerCompute, queryExecutor });
} catch (error) {
input.io?.stderr.write(`KTX MCP memory_ingest disabled: ${error instanceof Error ? error.message : String(error)}\n`);
}
return () =>
createDefaultKtxMcpServer({
name: 'ktx',
version: input.cliVersion,
userContext: { userId: 'local' },
contextTools: {
...contextTools,
...(memoryIngest ? { memoryIngest } : {}),
},
});
}

64
packages/cli/src/mcp-stdio-server.ts Normal file
View file

@ -0,0 +1,64 @@
import process from 'node:process';
import type { Readable, Writable } from 'node:stream';
import { loadKtxProject } from '@ktx/context/project';
import type { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
import type { KtxCliIo } from './cli-runtime.js';
import { createKtxMcpServerFactory } from './mcp-server-factory.js';
export interface RunKtxMcpStdioServerOptions {
projectDir: string;
cliVersion?: string;
io?: KtxCliIo;
createMcpServer?: () => McpServer;
loadProject?: typeof loadKtxProject;
stdin?: Readable;
stdout?: Writable;
}
export async function runKtxMcpStdioServer(options: RunKtxMcpStdioServerOptions): Promise<void> {
const project =
options.createMcpServer === undefined
? await (options.loadProject ?? loadKtxProject)({ projectDir: options.projectDir })
: undefined;
const protocolIo: KtxCliIo = {
stdout: { write() {} },
stderr: options.io?.stderr ?? { write() {} },
};
const createMcpServer =
options.createMcpServer ??
(await createKtxMcpServerFactory({
project: project!,
projectDir: options.projectDir,
cliVersion: options.cliVersion ?? '0.0.0-private',
io: protocolIo,
}));
const stdin = options.stdin ?? process.stdin;
const transport = new StdioServerTransport(stdin, options.stdout);
await new Promise<void>((resolve, reject) => {
let settled = false;
const settle = (callback: () => void) => {
if (settled) return;
settled = true;
stdin.off('end', closeTransport);
stdin.off('close', closeTransport);
callback();
};
const closeTransport = () => {
transport.close().catch((error: unknown) => {
settle(() => reject(error instanceof Error ? error : new Error(String(error))));
});
};
transport.onclose = () => settle(resolve);
transport.onerror = (error) => {
options.io?.stderr.write(`KTX MCP stdio transport error: ${error.message}\n`);
settle(() => reject(error));
};
stdin.once('end', closeTransport);
stdin.once('close', closeTransport);
createMcpServer().connect(transport).catch((error: unknown) => {
settle(() => reject(error instanceof Error ? error : new Error(String(error))));
});
});
}

2
packages/cli/src/memory-flow-tui.test.tsx
View file

@ -27,7 +27,7 @@ function replayInput(): MemoryFlowReplayInput {
{ unitKey: 'customers', target: 'sl', action: 'updated', key: 'orbit_demo.customers', summary: 'customer metrics', rawFiles: ['customers'], status: 'success' },
],
provenance: [{ rawPath: 'orders', artifactKind: 'wiki', artifactKey: 'wiki/orders.md', actionType: 'wiki_written' }],
transcripts: [{ unitKey: 'orders', path: '/tmp/t.jsonl', toolCallCount: 2, errorCount: 0, toolNames: ['read_raw_span', 'wiki_write'] }],
transcripts: [{ unitKey: 'orders', path: '/tmp/t.jsonl', toolCallCount: 2, errorCount: 0, toolNames: ['read_raw_span', 'memory_ingest'] }],
},
events: [
{ type: 'source_acquired', adapter: 'live-database', trigger: 'manual_resync', fileCount: 2 },

497
packages/cli/src/setup-agents.test.ts
View file

@ -2,6 +2,7 @@ import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { readKtxSetupState } from '@ktx/context/project';
import { strFromU8, unzipSync } from 'fflate';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import {
formatInstallSummary,
@ -24,6 +25,44 @@ function makeIo() {
};
}
async function readZipText(path: string, entry: string): Promise<string> {
const archive = unzipSync(new Uint8Array(await readFile(path)));
const content = archive[entry];
if (!content) throw new Error(`Missing zip entry: ${entry}`);
return strFromU8(content);
}
function captureEnvKeys(env: NodeJS.ProcessEnv, keys: readonly string[]): Record<string, string | undefined> {
const snapshot: Record<string, string | undefined> = {};
for (const key of keys) snapshot[key] = env[key];
return snapshot;
}
function clearEnvKeys(env: NodeJS.ProcessEnv, keys: readonly string[]): void {
for (const key of keys) delete env[key];
}
function captureKtxEnv(env: NodeJS.ProcessEnv): Record<string, string | undefined> {
const snapshot: Record<string, string | undefined> = {};
for (const key of Object.keys(env)) {
if (key.startsWith('KTX_')) snapshot[key] = env[key];
}
return snapshot;
}
function clearKtxEnv(env: NodeJS.ProcessEnv): void {
for (const key of Object.keys(env)) {
if (key.startsWith('KTX_')) delete env[key];
}
}
function restoreEnvKeys(env: NodeJS.ProcessEnv, snapshot: Record<string, string | undefined>): void {
for (const [key, value] of Object.entries(snapshot)) {
if (value === undefined) delete env[key];
else env[key] = value;
}
}
describe('setup agents', () => {
let tempDir: string;
@ -37,28 +76,54 @@ describe('setup agents', () => {
await rm(tempDir, { recursive: true, force: true });
});
it('plans project-scoped CLI and research files for every target', () => {
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'claude-code', scope: 'project', mode: 'cli' })).toEqual([
it('plans project-scoped MCP analytics files for every target', () => {
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'claude-code', scope: 'project', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'claude-desktop', scope: 'global', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.ktx/agents/claude/ktx-plugin-runner.sh'), role: 'launcher' },
{ kind: 'file', path: join(tempDir, '.ktx/agents/claude/ktx-plugin.zip'), role: 'claude-plugin' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'codex', scope: 'project', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'cursor', scope: 'project', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx-analytics.mdc'), role: 'analytics-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'opencode', scope: 'project', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.opencode/commands/ktx-analytics.md'), role: 'analytics-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'universal', scope: 'project', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
]);
});
it('plans project-scoped admin CLI files for every target when requested', () => {
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'claude-code', scope: 'project', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx-research/SKILL.md'), role: 'research-skill' },
{ kind: 'file', path: join(tempDir, '.claude/rules/ktx.md'), role: 'rule' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'codex', scope: 'project', mode: 'cli' })).toEqual([
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'codex', scope: 'project', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-research/SKILL.md'), role: 'research-skill' },
{ kind: 'file', path: join(tempDir, '.codex/instructions/ktx.md'), role: 'rule' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'cursor', scope: 'project', mode: 'cli' })).toEqual([
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'cursor', scope: 'project', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx-analytics.mdc'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx.mdc') },
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx-research.mdc'), role: 'research-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'opencode', scope: 'project', mode: 'cli' })).toEqual([
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'opencode', scope: 'project', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.opencode/commands/ktx-analytics.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.opencode/commands/ktx.md') },
{ kind: 'file', path: join(tempDir, '.opencode/commands/ktx-research.md'), role: 'research-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'universal', scope: 'project', mode: 'cli' })).toEqual([
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'universal', scope: 'project', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx/SKILL.md') },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-research/SKILL.md'), role: 'research-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'claude-desktop', scope: 'global', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.ktx/agents/claude/ktx-plugin-runner.sh'), role: 'launcher' },
{ kind: 'file', path: join(tempDir, '.ktx/agents/claude/ktx-plugin.zip'), role: 'claude-plugin' },
]);
});
@ -74,7 +139,7 @@ describe('setup agents', () => {
agents: true,
target: 'universal',
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
@ -82,7 +147,7 @@ describe('setup agents', () => {
).resolves.toEqual({
status: 'ready',
projectDir: tempDir,
installs: [{ target: 'universal', scope: 'project', mode: 'cli' }],
installs: [{ target: 'universal', scope: 'project', mode: 'mcp-cli' }],
});
await expect(stat(join(tempDir, '.agents/skills/ktx/SKILL.md'))).resolves.toBeDefined();
@ -99,13 +164,13 @@ describe('setup agents', () => {
expect(await readKtxAgentInstallManifest(tempDir)).toMatchObject({
version: 1,
projectDir: tempDir,
installs: [{ target: 'universal', scope: 'project', mode: 'cli' }],
installs: [{ target: 'universal', scope: 'project', mode: 'mcp-cli' }],
});
expect(await readKtxSetupState(tempDir)).toEqual({ completed_steps: ['agents'] });
expect(io.stderr()).toBe('');
});
it('installs the research skill from the runtime asset', async () => {
it('installs the analytics skill from the runtime asset', async () => {
const io = makeIo();
await expect(
@ -117,17 +182,20 @@ describe('setup agents', () => {
agents: true,
target: 'universal',
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
),
).resolves.toMatchObject({ status: 'ready' });
const researchSkill = await readFile(join(tempDir, '.agents/skills/ktx-research/SKILL.md'), 'utf-8');
expect(researchSkill).toContain('name: ktx-research');
expect(researchSkill).toContain('Always run `discover_data` before writing SQL.');
expect(researchSkill).toContain('Treat a `dictionary_search` miss as non-authoritative.');
const analyticsSkill = await readFile(join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), 'utf-8');
expect(analyticsSkill).toContain('name: ktx-analytics');
expect(analyticsSkill).toContain('Always run `discover_data` before writing SQL.');
expect(analyticsSkill).toContain('Treat a `dictionary_search` miss as non-authoritative.');
expect(analyticsSkill).toContain('memory_ingest');
expect(analyticsSkill).toContain('ARR is reported in cents');
expect(analyticsSkill).not.toContain(`memory_${'capture'}`);
});
it('writes PATH-independent launcher commands for skills', async () => {
@ -142,7 +210,7 @@ describe('setup agents', () => {
agents: true,
target: 'universal',
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
@ -170,7 +238,7 @@ describe('setup agents', () => {
agents: true,
target: 'claude-code',
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
@ -187,6 +255,279 @@ describe('setup agents', () => {
expect(io.stdout()).toContain('Run `ktx mcp start` to enable the configured KTX MCP server.');
});
it('prompts for MCP-first client agent connection mode in interactive setup', async () => {
const io = makeIo();
const prompts = {
select: vi.fn(async ({ message }: { message: string }) => (message.startsWith('Where') ? 'project' : 'mcp')),
multiselect: vi.fn(async () => ['claude-code']),
cancel: vi.fn(),
};
await expect(
runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'auto',
yes: false,
agents: true,
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
io.io,
{ prompts },
),
).resolves.toMatchObject({
status: 'ready',
installs: [{ target: 'claude-code', scope: 'project', mode: 'mcp' }],
});
expect(prompts.select).toHaveBeenCalledWith({
message: 'How should client agents connect to this KTX project?',
options: [
{ value: 'mcp', label: 'MCP tools + analytics skill' },
{ value: 'mcp-cli', label: 'MCP tools + analytics skill + admin CLI skill' },
],
});
expect(prompts.multiselect).toHaveBeenCalledWith(
expect.objectContaining({
options: expect.arrayContaining([{ value: 'claude-desktop', label: 'Claude Desktop' }]),
}),
);
});
it('prompts for global scope when every selected target supports it', async () => {
const home = await mkdtemp(join(tmpdir(), 'ktx-setup-agents-home-'));
const previousHome = process.env.HOME;
process.env.HOME = home;
try {
const io = makeIo();
const prompts = {
select: vi.fn(async ({ message }: { message: string }) =>
message.startsWith('Where should') ? 'global' : 'mcp',
),
multiselect: vi.fn(async () => ['claude-code']),
cancel: vi.fn(),
};
await expect(
runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'auto',
yes: false,
agents: true,
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
io.io,
{ prompts },
),
).resolves.toMatchObject({
status: 'ready',
installs: [{ target: 'claude-code', scope: 'global', mode: 'mcp' }],
});
expect(prompts.select).toHaveBeenCalledWith({
message: 'Where should KTX install supported agent config?',
options: [
{ value: 'project', label: 'Project' },
{ value: 'global', label: 'Global' },
],
});
} finally {
process.env.HOME = previousHome;
await rm(home, { recursive: true, force: true });
}
});
it('registers Claude Desktop MCP via claude_desktop_config.json and ships a skills-only plugin', async () => {
const home = await mkdtemp(join(tmpdir(), 'ktx-setup-agents-home-'));
const previousHome = process.env.HOME;
const envSnapshot = captureEnvKeys(process.env, ['OPENAI_API_KEY', 'ANTHROPIC_API_KEY']);
const ktxEnvSnapshot = captureKtxEnv(process.env);
process.env.HOME = home;
clearEnvKeys(process.env, ['OPENAI_API_KEY', 'ANTHROPIC_API_KEY']);
clearKtxEnv(process.env);
try {
const io = makeIo();
await expect(
runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'claude-desktop',
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
io.io,
),
).resolves.toMatchObject({
status: 'ready',
installs: [{ target: 'claude-desktop', scope: 'global', mode: 'mcp' }],
});
const pluginPath = join(tempDir, '.ktx/agents/claude/ktx-plugin.zip');
const launcherPath = join(tempDir, '.ktx/agents/claude/ktx-plugin-runner.sh');
await expect(stat(pluginPath)).resolves.toBeDefined();
const launcherStat = await stat(launcherPath);
expect(launcherStat.mode & 0o111).not.toBe(0);
const launcher = await readFile(launcherPath, 'utf-8');
expect(launcher).toContain('KTX_CLI_BIN=');
expect(launcher).toContain('.nvm/versions/node');
const configPath = join(home, 'Library/Application Support/Claude/claude_desktop_config.json');
const config = JSON.parse(await readFile(configPath, 'utf-8')) as {
mcpServers: { ktx: { command: string; args: string[]; env?: Record<string, string> } };
};
expect(config.mcpServers.ktx).toEqual({
command: launcherPath,
args: ['--project-dir', tempDir, 'mcp', 'stdio'],
});
expect(await readZipText(pluginPath, '.claude-plugin/plugin.json')).toContain('"name": "ktx"');
await expect(readZipText(pluginPath, '.mcp.json')).rejects.toThrow('Missing zip entry');
expect(await readZipText(pluginPath, 'skills/ktx-analytics/SKILL.md')).toContain('KTX Analytics Workflow');
const setupMd = await readZipText(pluginPath, 'SETUP.md');
expect(setupMd).not.toContain('ktx mcp start');
expect(setupMd).toContain('claude_desktop_config.json');
await expect(readZipText(pluginPath, 'skills/ktx/SKILL.md')).rejects.toThrow('Missing zip entry');
expect(io.stdout()).toContain('Claude plugin generated');
expect(io.stdout()).toContain('.ktx/agents/claude/ktx-plugin.zip');
expect(io.stdout()).toContain('KTX MCP server registered');
expect(io.stdout()).toContain('claude_desktop_config.json');
expect(io.stdout()).toContain('Restart Claude Desktop');
expect(io.stdout()).not.toContain('Run `ktx mcp start`');
} finally {
process.env.HOME = previousHome;
restoreEnvKeys(process.env, envSnapshot);
restoreEnvKeys(process.env, ktxEnvSnapshot);
await rm(home, { recursive: true, force: true });
}
});
it('captures KTX_*, OPENAI_API_KEY, and ANTHROPIC_API_KEY into the Claude Desktop MCP env block', async () => {
const home = await mkdtemp(join(tmpdir(), 'ktx-setup-agents-home-'));
const previousHome = process.env.HOME;
const envSnapshot = captureEnvKeys(process.env, [
'OPENAI_API_KEY',
'ANTHROPIC_API_KEY',
'KTX_LOG_LEVEL',
]);
const ktxEnvSnapshot = captureKtxEnv(process.env);
process.env.HOME = home;
clearKtxEnv(process.env);
process.env.OPENAI_API_KEY = 'sk-test-openai'; // pragma: allowlist secret
process.env.ANTHROPIC_API_KEY = 'sk-ant-test'; // pragma: allowlist secret
process.env.KTX_LOG_LEVEL = 'debug';
try {
const io = makeIo();
await runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'claude-desktop',
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
io.io,
);
const configPath = join(home, 'Library/Application Support/Claude/claude_desktop_config.json');
const config = JSON.parse(await readFile(configPath, 'utf-8')) as {
mcpServers: { ktx: { env?: Record<string, string> } };
};
expect(config.mcpServers.ktx.env).toEqual({
OPENAI_API_KEY: 'sk-test-openai', // pragma: allowlist secret
ANTHROPIC_API_KEY: 'sk-ant-test', // pragma: allowlist secret
KTX_LOG_LEVEL: 'debug',
});
} finally {
process.env.HOME = previousHome;
restoreEnvKeys(process.env, envSnapshot);
restoreEnvKeys(process.env, ktxEnvSnapshot);
await rm(home, { recursive: true, force: true });
}
});
it('includes the admin CLI skill in the Claude Desktop plugin zip when requested', async () => {
const home = await mkdtemp(join(tmpdir(), 'ktx-setup-agents-home-'));
const previousHome = process.env.HOME;
process.env.HOME = home;
try {
const io = makeIo();
await expect(
runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'claude-desktop',
scope: 'project',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
),
).resolves.toMatchObject({
status: 'ready',
installs: [{ target: 'claude-desktop', scope: 'global', mode: 'mcp-cli' }],
});
const pluginPath = join(tempDir, '.ktx/agents/claude/ktx-plugin.zip');
const adminSkill = await readZipText(pluginPath, 'skills/ktx/SKILL.md');
expect(adminSkill).toContain(`--project-dir ${tempDir}`);
expect(adminSkill).toContain('status --json');
expect(await readZipText(pluginPath, 'SETUP.md')).toContain('admin CLI skill');
await expect(readZipText(pluginPath, '.mcp.json')).rejects.toThrow('Missing zip entry');
} finally {
process.env.HOME = previousHome;
await rm(home, { recursive: true, force: true });
}
});
it('installs MCP client config and analytics skill without admin CLI files', async () => {
const io = makeIo();
await expect(
runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'claude-code',
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
io.io,
),
).resolves.toMatchObject({
status: 'ready',
installs: [{ target: 'claude-code', scope: 'project', mode: 'mcp' }],
});
const mcpJson = JSON.parse(await readFile(join(tempDir, '.mcp.json'), 'utf-8')) as {
mcpServers: { ktx: { type: string; url: string } };
};
expect(mcpJson.mcpServers.ktx).toEqual({ type: 'http', url: 'http://localhost:7878/mcp' });
await expect(stat(join(tempDir, '.claude/skills/ktx-analytics/SKILL.md'))).resolves.toBeDefined();
await expect(stat(join(tempDir, '.claude/skills/ktx/SKILL.md'))).rejects.toThrow();
await expect(stat(join(tempDir, '.claude/rules/ktx.md'))).rejects.toThrow();
});
it('writes Cursor project MCP config', async () => {
const io = makeIo();
@ -198,7 +539,7 @@ describe('setup agents', () => {
agents: true,
target: 'cursor',
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
@ -210,7 +551,7 @@ describe('setup agents', () => {
expect(cursorJson.mcpServers.ktx).toEqual({ url: 'http://localhost:7878/mcp' });
});
it('prints Codex and opencode snippets without mutating printed-only config files', async () => {
it('prints Codex, opencode, and universal snippets without mutating printed-only config files', async () => {
const codexIo = makeIo();
await runKtxSetupAgentsStep(
{
@ -220,7 +561,7 @@ describe('setup agents', () => {
agents: true,
target: 'codex',
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
codexIo.io,
@ -237,7 +578,7 @@ describe('setup agents', () => {
agents: true,
target: 'opencode',
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
opencodeIo.io,
@ -245,6 +586,23 @@ describe('setup agents', () => {
expect(opencodeIo.stdout()).toContain('"mcp"');
expect(opencodeIo.stdout()).toContain('"type": "remote"');
await expect(readFile(join(tempDir, 'opencode.json'), 'utf-8')).rejects.toThrow();
const universalIo = makeIo();
await runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'universal',
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
universalIo.io,
);
expect(universalIo.stdout()).toContain('Universal MCP endpoint:');
expect(universalIo.stdout()).toContain('http://localhost:7878/mcp');
});
it('uses MCP daemon state for port and token metadata without rendering literal tokens', async () => {
@ -280,7 +638,7 @@ describe('setup agents', () => {
agents: true,
target: 'claude-code',
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
@ -314,7 +672,7 @@ describe('setup agents', () => {
agents: true,
target: 'claude-code',
scope: 'local',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
@ -340,7 +698,7 @@ describe('setup agents', () => {
agents: true,
target: 'claude-code',
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
@ -355,6 +713,50 @@ describe('setup agents', () => {
await expect(readKtxAgentInstallManifest(tempDir)).resolves.toEqual(null);
});
it('removes generated Claude Desktop plugin from the manifest', async () => {
const home = await mkdtemp(join(tmpdir(), 'ktx-setup-agents-home-'));
const previousHome = process.env.HOME;
process.env.HOME = home;
try {
const io = makeIo();
await runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'claude-desktop',
scope: 'project',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
);
const pluginPath = join(tempDir, '.ktx/agents/claude/ktx-plugin.zip');
const launcherPath = join(tempDir, '.ktx/agents/claude/ktx-plugin-runner.sh');
const configPath = join(home, 'Library/Application Support/Claude/claude_desktop_config.json');
await expect(stat(pluginPath)).resolves.toBeDefined();
await expect(stat(launcherPath)).resolves.toBeDefined();
const beforeConfig = JSON.parse(await readFile(configPath, 'utf-8')) as {
mcpServers: Record<string, unknown>;
};
expect(beforeConfig.mcpServers.ktx).toBeDefined();
await expect(removeKtxAgentInstall(tempDir, io.io)).resolves.toBe(0);
await expect(stat(pluginPath)).rejects.toThrow();
await expect(stat(launcherPath)).rejects.toThrow();
const afterConfig = JSON.parse(await readFile(configPath, 'utf-8')) as {
mcpServers: Record<string, unknown>;
};
expect(afterConfig.mcpServers.ktx).toBeUndefined();
await expect(readKtxAgentInstallManifest(tempDir)).resolves.toEqual(null);
} finally {
process.env.HOME = previousHome;
await rm(home, { recursive: true, force: true });
}
});
it('treats cancel as skip in interactive mode', async () => {
const io = makeIo();
const prompts = {
@ -371,7 +773,7 @@ describe('setup agents', () => {
yes: false,
agents: true,
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
@ -383,7 +785,7 @@ describe('setup agents', () => {
it('explains how to select multiple agent targets in interactive mode', async () => {
const io = makeIo();
const prompts = {
select: vi.fn(async () => 'cli'),
select: vi.fn(async () => 'mcp-cli'),
multiselect: vi.fn(async () => ['back']),
cancel: vi.fn(),
};
@ -396,7 +798,7 @@ describe('setup agents', () => {
yes: false,
agents: true,
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
@ -423,7 +825,7 @@ describe('setup agents', () => {
agents: true,
target: 'claude-code',
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
@ -432,21 +834,28 @@ describe('setup agents', () => {
const output = io.stdout();
expect(output).toContain('Agent integration complete');
expect(output).toContain('Claude Code');
expect(output).toContain('+ Skill installed — teaches your agent which KTX commands to run');
expect(output).toContain('+ Analytics skill installed — teaches your agent the KTX MCP analytics workflow');
expect(output).toContain('.claude/skills/ktx-analytics/SKILL.md');
expect(output).toContain('+ Skill installed — teaches admin agents which KTX CLI commands to run');
expect(output).toContain('.claude/skills/ktx/SKILL.md');
expect(output).toContain('+ Rule installed — tells your agent when to use KTX');
expect(output).toContain('+ Rule installed — tells admin agents when to use KTX CLI');
expect(output).toContain('.claude/rules/ktx.md');
});
it('formats summary with relative paths for project scope', () => {
const summary = formatInstallSummary(
[{ target: 'cursor', scope: 'project', mode: 'cli' }],
[{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx.mdc') }],
[{ target: 'cursor', scope: 'project', mode: 'mcp-cli' }],
[
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx-analytics.mdc'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx.mdc') },
],
tempDir,
);
expect(summary).toContain('Cursor');
expect(summary).toContain('+ Rule installed — tells your agent when to use KTX');
expect(summary).toContain('+ Analytics skill installed — teaches your agent the KTX MCP analytics workflow');
expect(summary).toContain('.cursor/rules/ktx-analytics.mdc');
expect(summary).toContain('+ Rule installed — tells admin agents when to use KTX CLI');
expect(summary).toContain('.cursor/rules/ktx.mdc');
expect(summary).not.toContain(tempDir);
});
@ -454,12 +863,14 @@ describe('setup agents', () => {
it('formats summary with multiple agent targets', () => {
const summary = formatInstallSummary(
[
{ target: 'claude-code', scope: 'project', mode: 'cli' },
{ target: 'codex', scope: 'project', mode: 'cli' },
{ target: 'claude-code', scope: 'project', mode: 'mcp-cli' },
{ target: 'codex', scope: 'project', mode: 'mcp-cli' },
],
[
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(tempDir, '.claude/rules/ktx.md'), role: 'rule' },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(tempDir, '.codex/instructions/ktx.md'), role: 'rule' },
],
@ -467,9 +878,11 @@ describe('setup agents', () => {
);
expect(summary).toContain('Claude Code');
expect(summary).toContain('+ Skill installed — teaches your agent which KTX commands to run');
expect(summary).toContain('+ Rule installed — tells your agent when to use KTX');
expect(summary).toContain('+ Analytics skill installed — teaches your agent the KTX MCP analytics workflow');
expect(summary).toContain('+ Skill installed — teaches admin agents which KTX CLI commands to run');
expect(summary).toContain('+ Rule installed — tells admin agents when to use KTX CLI');
expect(summary).toContain('Codex');
expect(summary).toContain('.agents/skills/ktx-analytics/SKILL.md');
expect(summary).toContain('.agents/skills/ktx/SKILL.md');
});
});

460
packages/cli/src/setup-agents.ts
View file

@ -1,5 +1,5 @@
import { existsSync } from 'node:fs';
import { mkdir, readFile, rm, writeFile } from 'node:fs/promises';
import { chmod, mkdir, readFile, rm, writeFile } from 'node:fs/promises';
import { dirname, join, relative, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import {
@ -7,17 +7,20 @@ import {
markKtxSetupStateStepComplete,
serializeKtxProjectConfig,
} from '@ktx/context/project';
import { strToU8, zipSync } from 'fflate';
import type { KtxCliIo } from './cli-runtime.js';
import { bold, dim, green } from './io/symbols.js';
import { withMultiselectNavigation } from './prompt-navigation.js';
import {
createKtxSetupPromptAdapter,
createKtxSetupUiAdapter,
type KtxSetupPromptOption,
} from './setup-prompts.js';
import { readKtxMcpDaemonStatus } from './managed-mcp-daemon.js';
export type KtxAgentTarget = 'claude-code' | 'codex' | 'cursor' | 'opencode' | 'universal';
export type KtxAgentTarget = 'claude-code' | 'claude-desktop' | 'codex' | 'cursor' | 'opencode' | 'universal';
export type KtxAgentScope = 'project' | 'global' | 'local';
export type KtxAgentInstallMode = 'cli';
export type KtxAgentInstallMode = 'mcp' | 'mcp-cli';
export interface KtxSetupAgentsArgs {
projectDir: string;
@ -47,7 +50,7 @@ export interface KtxAgentInstallManifest {
installedAt: string;
installs: Array<{ target: KtxAgentTarget; scope: KtxAgentScope; mode: KtxAgentInstallMode }>;
entries: Array<
| { kind: 'file'; path: string; role?: 'skill' | 'rule' | 'research-skill' }
| { kind: 'file'; path: string; role?: 'skill' | 'rule' | 'analytics-skill' | 'claude-plugin' | 'launcher' }
| { kind: 'json-key'; path: string; jsonPath: string[] }
>;
}
@ -169,6 +172,14 @@ function opencodeSnippet(endpoint: KtxMcpEndpointInfo): string {
);
}
function universalMcpSnippet(endpoint: KtxMcpEndpointInfo): string {
return [
'Universal MCP endpoint:',
endpoint.url,
...(endpoint.tokenAuth ? ['Header: Authorization: Bearer ${KTX_MCP_TOKEN}'] : []),
].join('\n');
}
function claudeConfigPath(projectDir: string, scope: KtxAgentScope): { path: string; jsonPath: string[] } {
const home = process.env.HOME ?? '';
if (scope === 'global') {
@ -188,16 +199,63 @@ function cursorConfigPath(projectDir: string, scope: KtxAgentScope): { path: str
};
}
function claudeDesktopConfigPath(): { path: string; jsonPath: string[] } {
const home = process.env.HOME ?? '';
const path =
process.platform === 'win32'
? join(process.env.APPDATA ?? join(home, 'AppData/Roaming'), 'Claude/claude_desktop_config.json')
: join(home, 'Library/Application Support/Claude/claude_desktop_config.json');
return { path, jsonPath: ['mcpServers', 'ktx'] };
}
const CLAUDE_DESKTOP_FORWARDED_ENV_KEYS = ['OPENAI_API_KEY', 'ANTHROPIC_API_KEY'] as const;
export function collectClaudeDesktopForwardedEnv(source: NodeJS.ProcessEnv): Record<string, string> {
const captured: Record<string, string> = {};
for (const [key, value] of Object.entries(source)) {
if (value === undefined || value === '') continue;
if (key.startsWith('KTX_') || (CLAUDE_DESKTOP_FORWARDED_ENV_KEYS as readonly string[]).includes(key)) {
captured[key] = value;
}
}
return captured;
}
function claudeDesktopMcpEntry(input: {
launcherPath: string;
projectDir: string;
env?: NodeJS.ProcessEnv;
}): Record<string, unknown> {
const captured = collectClaudeDesktopForwardedEnv(input.env ?? process.env);
return {
command: input.launcherPath,
args: ['--project-dir', input.projectDir, 'mcp', 'stdio'],
...(Object.keys(captured).length > 0 ? { env: captured } : {}),
};
}
async function installMcpClientConfig(input: {
projectDir: string;
target: KtxAgentTarget;
scope: KtxAgentScope;
}): Promise<KtxMcpClientInstallResult> {
const endpoint = await resolveMcpEndpoint(input.projectDir);
const entries: InstallEntry[] = [];
const snippets: string[] = [];
const notices: string[] = [];
if (input.target === 'claude-desktop') {
const config = claudeDesktopConfigPath();
const launcherPath = claudeDesktopLauncherPath(input.projectDir);
await writeJsonKey(
config.path,
config.jsonPath,
claudeDesktopMcpEntry({ launcherPath, projectDir: input.projectDir }),
);
entries.push({ kind: 'json-key', path: config.path, jsonPath: config.jsonPath });
return { entries, snippets, notices };
}
const endpoint = await resolveMcpEndpoint(input.projectDir);
if (!endpoint.running) {
notices.push('Run `ktx mcp start` to enable the configured KTX MCP server.');
}
@ -213,74 +271,141 @@ async function installMcpClientConfig(input: {
} else if (input.target === 'codex') {
snippets.push(`Codex MCP snippet for ~/.codex/config.toml:\n${codexSnippet(endpoint)}`);
} else if (input.target === 'opencode') {
const path = input.scope === 'global' ? '~/.config/opencode/opencode.json' : relative(input.projectDir, join(input.projectDir, 'opencode.json'));
const path =
input.scope === 'global'
? '~/.config/opencode/opencode.json'
: relative(input.projectDir, join(input.projectDir, 'opencode.json'));
snippets.push(`opencode MCP snippet for ${path}:\n${opencodeSnippet(endpoint)}`);
} else if (input.target === 'universal') {
snippets.push(universalMcpSnippet(endpoint));
}
return { entries, snippets, notices };
}
function plannedMcpJsonEntries(input: {
projectDir: string;
target: KtxAgentTarget;
scope: KtxAgentScope;
}): InstallEntry[] {
if (input.target === 'claude-code') {
const config = claudeConfigPath(input.projectDir, input.scope);
return [{ kind: 'json-key', path: config.path, jsonPath: config.jsonPath }];
}
if (input.target === 'claude-desktop') {
const config = claudeDesktopConfigPath();
return [{ kind: 'json-key', path: config.path, jsonPath: config.jsonPath }];
}
if (input.target === 'cursor') {
const config = cursorConfigPath(input.projectDir, input.scope);
return [{ kind: 'json-key', path: config.path, jsonPath: config.jsonPath }];
}
return [];
}
export function agentInstallManifestPath(projectDir: string): string {
return join(resolve(projectDir), '.ktx/agents/install-manifest.json');
}
function claudeDesktopPluginPath(projectDir: string): string {
return join(resolve(projectDir), '.ktx/agents/claude/ktx-plugin.zip');
}
function claudeDesktopLauncherPath(projectDir: string): string {
return join(resolve(projectDir), '.ktx/agents/claude/ktx-plugin-runner.sh');
}
export function plannedKtxAgentFiles(input: {
projectDir: string;
target: KtxAgentTarget;
scope: KtxAgentScope;
mode: KtxAgentInstallMode;
}): InstallEntry[] {
const withAdminCli = input.mode === 'mcp-cli';
if (input.scope === 'global') {
if (input.target === 'claude-code') {
const home = process.env.HOME ?? '';
return [
{ kind: 'file', path: join(home, '.claude/skills/ktx/SKILL.md'), role: 'skill' as const },
{ kind: 'file', path: join(home, '.claude/skills/ktx-research/SKILL.md'), role: 'research-skill' as const },
{ kind: 'file', path: join(home, '.claude/rules/ktx.md'), role: 'rule' as const },
{ kind: 'file', path: join(home, '.claude/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' as const },
...(withAdminCli
? [
{ kind: 'file' as const, path: join(home, '.claude/skills/ktx/SKILL.md'), role: 'skill' as const },
{ kind: 'file' as const, path: join(home, '.claude/rules/ktx.md'), role: 'rule' as const },
]
: []),
];
}
if (input.target === 'codex') {
const codexHome = process.env.CODEX_HOME ?? join(process.env.HOME ?? '', '.codex');
return [
{ kind: 'file', path: join(codexHome, 'skills/ktx/SKILL.md'), role: 'skill' as const },
{ kind: 'file', path: join(codexHome, 'skills/ktx-research/SKILL.md'), role: 'research-skill' as const },
{ kind: 'file', path: join(codexHome, 'instructions/ktx.md'), role: 'rule' as const },
{ kind: 'file', path: join(codexHome, 'skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' as const },
...(withAdminCli
? [
{ kind: 'file' as const, path: join(codexHome, 'skills/ktx/SKILL.md'), role: 'skill' as const },
{ kind: 'file' as const, path: join(codexHome, 'instructions/ktx.md'), role: 'rule' as const },
]
: []),
];
}
if (input.target === 'cursor' || input.target === 'opencode') {
return [];
}
if (input.target === 'claude-desktop') {
return [
{ kind: 'file', path: claudeDesktopLauncherPath(input.projectDir), role: 'launcher' as const },
{ kind: 'file', path: claudeDesktopPluginPath(input.projectDir), role: 'claude-plugin' as const },
];
}
throw new Error(`Global ${input.target} installation is not supported; omit --global.`);
}
const root = resolve(input.projectDir);
const analyticsEntries: Partial<Record<KtxAgentTarget, InstallEntry[]>> = {
'claude-code': [
{ kind: 'file', path: join(root, '.claude/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
],
codex: [
{ kind: 'file', path: join(root, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
],
cursor: [
{ kind: 'file', path: join(root, '.cursor/rules/ktx-analytics.mdc'), role: 'analytics-skill' },
],
opencode: [
{ kind: 'file', path: join(root, '.opencode/commands/ktx-analytics.md'), role: 'analytics-skill' },
],
universal: [
{ kind: 'file', path: join(root, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
],
'claude-desktop': [],
};
const cliEntries: Partial<Record<KtxAgentTarget, InstallEntry[]>> = {
'claude-code': [
{ kind: 'file', path: join(root, '.claude/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(root, '.claude/skills/ktx-research/SKILL.md'), role: 'research-skill' },
],
codex: [
{ kind: 'file', path: join(root, '.agents/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(root, '.agents/skills/ktx-research/SKILL.md'), role: 'research-skill' },
],
cursor: [
{ kind: 'file', path: join(root, '.cursor/rules/ktx.mdc') },
{ kind: 'file', path: join(root, '.cursor/rules/ktx-research.mdc'), role: 'research-skill' },
],
opencode: [
{ kind: 'file', path: join(root, '.opencode/commands/ktx.md') },
{ kind: 'file', path: join(root, '.opencode/commands/ktx-research.md'), role: 'research-skill' },
],
universal: [
{ kind: 'file', path: join(root, '.agents/skills/ktx/SKILL.md') },
{ kind: 'file', path: join(root, '.agents/skills/ktx-research/SKILL.md'), role: 'research-skill' },
],
'claude-desktop': [],
};
const ruleEntries: Partial<Record<KtxAgentTarget, InstallEntry>> = {
'claude-code': { kind: 'file', path: join(root, '.claude/rules/ktx.md'), role: 'rule' },
codex: { kind: 'file', path: join(root, '.codex/instructions/ktx.md'), role: 'rule' },
};
return [...(cliEntries[input.target] ?? []), ruleEntries[input.target]].filter(
return [
...(analyticsEntries[input.target] ?? []),
...(withAdminCli ? (cliEntries[input.target] ?? []) : []),
...(withAdminCli ? [ruleEntries[input.target]] : []),
].filter(
(entry): entry is InstallEntry => entry !== undefined,
);
}
@ -292,8 +417,8 @@ function ktxCliLauncher(): KtxCliLauncher {
};
}
async function readResearchSkillContent(): Promise<string> {
const path = fileURLToPath(new URL('./skills/research/SKILL.md', import.meta.url));
async function readAnalyticsSkillContent(): Promise<string> {
const path = fileURLToPath(new URL('./skills/analytics/SKILL.md', import.meta.url));
const content = await readFile(path, 'utf-8');
return content.endsWith('\n') ? content : `${content}\n`;
}
@ -305,6 +430,10 @@ function shellQuote(value: string): string {
return `'${value.replaceAll("'", "'\\''")}'`;
}
function shellScriptQuote(value: string): string {
return `'${value.replaceAll("'", "'\\''")}'`;
}
function ktxCommandLine(launcher: KtxCliLauncher, args: string[]): string {
return [launcher.command, ...launcher.args, ...args].map(shellQuote).join(' ');
}
@ -320,11 +449,14 @@ function cliInstructionContent(input: { projectDir: string; launcher: KtxCliLaun
'',
'# KTX Local Context',
'',
'This is an admin/developer CLI helper. End-user data agents should use the KTX MCP tools when available.',
'',
`Use this project with \`--project-dir ${input.projectDir}\`.`,
'Commands are pinned to the local KTX CLI path that created this file, so agents do not need `ktx` in PATH.',
'If the CLI path no longer exists after moving this checkout or reinstalling KTX, rerun `ktx setup --agents`.',
'',
'Agents must not print secrets, credential references, environment variable values, or file contents from `.ktx/secrets`.',
'Agents must not print secrets, credential references, environment variable values, or file contents from ' +
'`.ktx/secrets`.',
'',
'Available commands:',
'',
@ -352,9 +484,132 @@ function cliInstructionContent(input: { projectDir: string; launcher: KtxCliLaun
].join('\n');
}
function claudePluginJsonContent(): string {
return `${JSON.stringify(
{
name: 'ktx',
version: '0.0.0-local',
description: 'KTX analytics workflow guidance and local MCP tools.',
},
null,
2,
)}\n`;
}
function claudePluginVersionContent(): string {
return `${JSON.stringify({ version: '0.0.0-local' }, null, 2)}\n`;
}
function claudePluginSetupContent(input: { projectDir: string; withAdminCli: boolean }): string {
return [
'# KTX Claude Plugin',
'',
'Install this plugin ZIP from Claude Desktop to load the KTX analytics skill.',
'',
`KTX project: \`${input.projectDir}\``,
'',
'Included:',
'',
'- `ktx-analytics` skill for the MCP analytics workflow',
...(input.withAdminCli ? ['- `ktx` admin CLI skill for KTX maintenance commands'] : []),
'',
'The KTX MCP server is registered separately in `claude_desktop_config.json` by `ktx setup` and runs as a local stdio child of Claude Desktop — no daemon to start.',
'',
'If this checkout or project directory moves, rerun `ktx setup --agents` and reinstall the regenerated plugin.',
'',
].join('\n');
}
function claudePluginLauncherContent(input: { launcher: KtxCliLauncher }): string {
const binPath = input.launcher.args[0];
if (!binPath) {
throw new Error('Expected KTX CLI launcher to include a bin path.');
}
const candidates = [
input.launcher.command,
'/opt/homebrew/bin/node',
'/usr/local/bin/node',
'/usr/bin/node',
];
return [
'#!/bin/sh',
'set -eu',
'',
`KTX_CLI_BIN=${shellScriptQuote(binPath)}`,
'',
'run_with_node() {',
' node_bin=$1',
' shift',
' exec "$node_bin" "$KTX_CLI_BIN" "$@"',
'}',
'',
'if [ -n "${KTX_NODE:-}" ] && [ -x "${KTX_NODE:-}" ]; then',
' run_with_node "$KTX_NODE" "$@"',
'fi',
'',
'if [ -x "$HOME/.volta/bin/node" ]; then',
' run_with_node "$HOME/.volta/bin/node" "$@"',
'fi',
'',
...candidates.map((candidate) =>
[
`if [ -x ${shellScriptQuote(candidate)} ]; then`,
` run_with_node ${shellScriptQuote(candidate)} "$@"`,
'fi',
].join('\n'),
),
'',
'for candidate in "$HOME"/.nvm/versions/node/*/bin/node; do',
' if [ -x "$candidate" ]; then',
' run_with_node "$candidate" "$@"',
' fi',
'done',
'',
'if command -v node >/dev/null 2>&1; then',
' run_with_node "$(command -v node)" "$@"',
'fi',
'',
'echo "KTX plugin could not find Node.js. Set KTX_NODE to a Node executable and reinstall the plugin." >&2',
'exit 127',
'',
].join('\n');
}
async function writeClaudeDesktopPlugin(input: {
projectDir: string;
path: string;
mode: KtxAgentInstallMode;
launcher: KtxCliLauncher;
}): Promise<void> {
const withAdminCli = input.mode === 'mcp-cli';
const files: Record<string, Uint8Array> = {
'.claude-plugin/plugin.json': strToU8(claudePluginJsonContent()),
'version.json': strToU8(claudePluginVersionContent()),
'skills/ktx-analytics/SKILL.md': strToU8(await readAnalyticsSkillContent()),
'SETUP.md': strToU8(claudePluginSetupContent({ projectDir: input.projectDir, withAdminCli })),
};
if (withAdminCli) {
files['skills/ktx/SKILL.md'] = strToU8(
cliInstructionContent({ projectDir: input.projectDir, launcher: input.launcher }),
);
}
await mkdir(dirname(input.path), { recursive: true });
await writeFile(input.path, Buffer.from(zipSync(files)));
}
async function writeClaudeDesktopLauncher(input: {
path: string;
launcher: KtxCliLauncher;
}): Promise<void> {
await mkdir(dirname(input.path), { recursive: true });
await writeFile(input.path, claudePluginLauncherContent({ launcher: input.launcher }), 'utf-8');
await chmod(input.path, 0o755);
}
function ruleInstructionContent(input: { projectDir: string }): string {
return [
`Use the \`ktx\` CLI to query local semantic context and wiki knowledge for this project (\`--project-dir ${input.projectDir}\`).`,
`Use the \`ktx\` CLI to query local semantic context and wiki knowledge for this project ` +
`(\`--project-dir ${input.projectDir}\`).`,
'',
'Use when the user asks about data schemas, metrics, dimensions, database structure, or wants to run SQL queries.',
'',
@ -390,7 +645,9 @@ async function writeManifest(projectDir: string, manifest: KtxAgentInstallManife
}
function entryKey(entry: InstallEntry): string {
return entry.kind === 'json-key' ? `${entry.kind}:${entry.path}:${entry.jsonPath.join('.')}` : `${entry.kind}:${entry.path}`;
return entry.kind === 'json-key'
? `${entry.kind}:${entry.path}:${entry.jsonPath.join('.')}`
: `${entry.kind}:${entry.path}`;
}
function mergeManifest(
@ -455,6 +712,7 @@ function createPromptAdapter(): KtxSetupAgentsPromptAdapter {
const targetDisplayNames: Record<KtxAgentTarget, string> = {
'claude-code': 'Claude Code',
'claude-desktop': 'Claude Desktop',
codex: 'Codex',
cursor: 'Cursor',
opencode: 'OpenCode',
@ -463,12 +721,25 @@ const targetDisplayNames: Record<KtxAgentTarget, string> = {
const fileEntryLabels: Record<KtxAgentTarget, string> = {
'claude-code': 'Skill installed',
'claude-desktop': 'Skill installed',
codex: 'Skill installed',
cursor: 'Rule installed',
opencode: 'Command installed',
universal: 'Skill installed',
};
function mcpEntryLabel(entry: Extract<InstallEntry, { kind: 'json-key' }>): string {
return `MCP config installed — connects client agents to KTX MCP tools (${entry.jsonPath.join('.')})`;
}
function targetSupportsGlobalScope(target: KtxAgentTarget): boolean {
return target === 'claude-code' || target === 'codex';
}
function effectiveInstallScope(target: KtxAgentTarget, requestedScope: KtxAgentScope): KtxAgentScope {
return target === 'claude-desktop' ? 'global' : requestedScope;
}
export function formatInstallSummary(
installs: Array<{ target: KtxAgentTarget; scope: KtxAgentScope; mode: KtxAgentInstallMode }>,
entries: InstallEntry[],
@ -486,11 +757,21 @@ export function formatInstallSummary(
entries.filter((entry) => entry.kind === 'file' && plannedFilePaths.has(entry.path)),
);
}
const mcpEntriesByTarget = new Map<KtxAgentTarget, InstallEntry[]>();
for (const install of installs) {
const plannedMcpKeys = new Set(plannedMcpJsonEntries({ projectDir, ...install }).map(entryKey));
mcpEntriesByTarget.set(
install.target,
entries.filter((entry) => entry.kind === 'json-key' && plannedMcpKeys.has(entryKey(entry))),
);
}
const fileHints: Record<string, string> = {
skill: 'teaches your agent which KTX commands to run',
rule: 'tells your agent when to use KTX',
'research-skill': 'teaches your agent the KTX MCP research workflow',
skill: 'teaches admin agents which KTX CLI commands to run',
rule: 'tells admin agents when to use KTX CLI',
'analytics-skill': 'teaches your agent the KTX MCP analytics workflow',
'claude-plugin': 'bundles KTX skills for Claude Desktop (MCP server is registered in claude_desktop_config.json)',
launcher: 'runs the local KTX CLI with an available Node.js for Claude Desktop',
};
const lines: string[] = [];
@ -498,16 +779,34 @@ export function formatInstallSummary(
const targetEntries = entriesByTarget.get(install.target) ?? [];
lines.push(` ${targetDisplayNames[install.target]}`);
for (const entry of targetEntries) {
const displayPath =
install.scope === 'global' ? entry.path : relative(projectDir, entry.path);
if (entry.kind === 'file') {
const isRule = entry.role === 'rule' || fileEntryLabels[install.target] === 'Rule installed';
const label = entry.role === 'research-skill' ? 'Research skill installed' : isRule ? 'Rule installed' : fileEntryLabels[install.target];
const isRule = entry.role === 'rule' || (!entry.role && fileEntryLabels[install.target] === 'Rule installed');
const label =
entry.role === 'analytics-skill'
? 'Analytics skill installed'
: entry.role === 'claude-plugin'
? 'Claude plugin generated'
: entry.role === 'launcher'
? 'Launcher installed'
: isRule
? 'Rule installed'
: fileEntryLabels[install.target];
const hint = fileHints[isRule ? 'rule' : (entry.role ?? 'skill')] ?? '';
lines.push(` + ${label} — ${hint}`);
lines.push(` ${displayPath}`);
if (entry.role !== 'claude-plugin') {
const displayPath =
install.scope === 'global' ? entry.path : relative(projectDir, entry.path);
lines.push(` ${displayPath}`);
}
}
}
for (const entry of mcpEntriesByTarget
.get(install.target)
?.filter((entry): entry is Extract<InstallEntry, { kind: 'json-key' }> => entry.kind === 'json-key') ?? []) {
const displayPath = install.scope === 'global' ? entry.path : relative(projectDir, entry.path);
lines.push(` + ${mcpEntryLabel(entry)}`);
lines.push(` ${displayPath}`);
}
}
return lines.join('\n');
}
@ -522,11 +821,24 @@ async function installTarget(input: {
const launcher = ktxCliLauncher();
for (const entry of entries) {
if (entry.kind !== 'file') continue;
if (entry.role === 'launcher') {
await writeClaudeDesktopLauncher({ path: entry.path, launcher });
continue;
}
if (entry.role === 'claude-plugin') {
await writeClaudeDesktopPlugin({
projectDir: input.projectDir,
path: entry.path,
mode: input.mode,
launcher,
});
continue;
}
const content =
entry.role === 'rule'
? ruleInstructionContent({ projectDir: input.projectDir })
: entry.role === 'research-skill'
? await readResearchSkillContent()
: entry.role === 'analytics-skill'
? await readAnalyticsSkillContent()
: cliInstructionContent({ projectDir: input.projectDir, launcher });
await mkdir(dirname(entry.path), { recursive: true });
await writeFile(entry.path, content, 'utf-8');
@ -558,14 +870,13 @@ export async function runKtxSetupAgentsStep(
args.inputMode === 'disabled'
? args.mode
: ((await prompts.select({
message: 'How should agents use this KTX project?',
message: 'How should client agents connect to this KTX project?',
options: [
{ value: 'cli', label: 'CLI tools and skills' },
{ value: 'skip', label: 'Skip' },
{ value: 'mcp', label: 'MCP tools + analytics skill' },
{ value: 'mcp-cli', label: 'MCP tools + analytics skill + admin CLI skill' },
],
})) as KtxAgentInstallMode | 'skip' | 'back');
})) as KtxAgentInstallMode | 'back');
if (mode === 'back') return { status: 'skipped', projectDir: args.projectDir };
if (mode === 'skip') return { status: 'skipped', projectDir: args.projectDir };
const targets =
args.target !== undefined
@ -576,6 +887,7 @@ export async function runKtxSetupAgentsStep(
message: withMultiselectNavigation('Which agent targets should KTX install?'),
options: [
{ value: 'claude-code', label: 'Claude Code' },
{ value: 'claude-desktop', label: 'Claude Desktop' },
{ value: 'codex', label: 'Codex' },
{ value: 'cursor', label: 'Cursor' },
{ value: 'opencode', label: 'OpenCode' },
@ -589,26 +901,80 @@ export async function runKtxSetupAgentsStep(
return { status: 'missing-input', projectDir: args.projectDir };
}
const installs = targets.map((target) => ({ target, scope: args.scope, mode }));
const scopeTargets = targets.filter((target) => target !== 'claude-desktop');
const selectedScope =
args.inputMode !== 'disabled' &&
args.scope === 'project' &&
scopeTargets.length > 0 &&
scopeTargets.every(targetSupportsGlobalScope)
? ((await prompts.select({
message: 'Where should KTX install supported agent config?',
options: [
{ value: 'project', label: 'Project' },
{ value: 'global', label: 'Global' },
],
})) as KtxAgentScope | 'back')
: args.scope;
if (selectedScope === 'back') return { status: 'back', projectDir: args.projectDir };
const installs = targets.map((target) => ({ target, scope: effectiveInstallScope(target, selectedScope), mode }));
const entries: InstallEntry[] = [];
const snippets: string[] = [];
const notices = new Set<string>();
let claudeDesktopTutorial: string | undefined;
try {
for (const install of installs) {
entries.push(...(await installTarget({ projectDir: args.projectDir, ...install })));
const mcpResult = await installMcpClientConfig({ projectDir: args.projectDir, target: install.target, scope: install.scope });
const targetEntries = await installTarget({ projectDir: args.projectDir, ...install });
entries.push(...targetEntries);
const mcpResult = await installMcpClientConfig({
projectDir: args.projectDir,
target: install.target,
scope: install.scope,
});
entries.push(...mcpResult.entries);
for (const snippet of mcpResult.snippets) snippets.push(snippet);
for (const notice of mcpResult.notices) notices.add(notice);
if (install.target === 'claude-desktop') {
const pluginEntry = targetEntries.find(
(entry): entry is Extract<InstallEntry, { kind: 'file' }> =>
entry.kind === 'file' && entry.role === 'claude-plugin',
);
const pluginPath = pluginEntry?.path ?? '';
const configPath = claudeDesktopConfigPath().path;
claudeDesktopTutorial = [
`${green('✓')} ${bold('KTX MCP server registered')}`,
` ${dim(configPath)}`,
'',
bold('1. Restart Claude Desktop'),
' Quit and reopen so it picks up the new MCP server.',
'',
bold('2. Install the KTX plugin'),
' Open Claude Desktop → Settings → Plugins and install from file:',
` 📦 ${dim(pluginPath)}`,
].join('\n');
}
}
await writeManifest(args.projectDir, mergeManifest(args.projectDir, await readKtxAgentInstallManifest(args.projectDir), installs, entries));
await writeManifest(
args.projectDir,
mergeManifest(args.projectDir, await readKtxAgentInstallManifest(args.projectDir), installs, entries),
);
await markAgentsComplete(args.projectDir);
io.stdout.write(`\nAgent integration complete\n\n${formatInstallSummary(installs, entries, args.projectDir)}\n`);
for (const snippet of snippets) {
io.stdout.write(`\n${snippet}\n`);
const setupUi = createKtxSetupUiAdapter();
setupUi.note(
formatInstallSummary(installs, entries, args.projectDir),
'Agent integration complete',
io,
);
if (claudeDesktopTutorial) {
setupUi.note(claudeDesktopTutorial, 'Finish Claude Desktop setup', io, {
format: (line) => line,
});
}
for (const notice of notices) {
io.stdout.write(`\n${notice}\n`);
const nextStepBlocks: string[] = [];
for (const notice of notices) nextStepBlocks.push(notice);
for (const snippet of snippets) nextStepBlocks.push(snippet);
if (nextStepBlocks.length > 0) {
setupUi.note(nextStepBlocks.join('\n\n'), 'Next steps', io, { format: bold });
}
return { status: 'ready', projectDir: args.projectDir, installs };
} catch (error) {

2
packages/cli/src/setup-demo-tour.test.ts
View file

@ -184,7 +184,7 @@ describe('runDemoTour', () => {
const mockAgents = vi.fn().mockResolvedValue({
status: 'ready',
projectDir: '/tmp/test',
installs: [{ target: 'claude-code', scope: 'project', mode: 'cli' }],
installs: [{ target: 'claude-code', scope: 'project', mode: 'mcp-cli' }],
} satisfies KtxSetupAgentsResult);
const navigation = vi.fn().mockResolvedValue('forward');

2
packages/cli/src/setup-demo-tour.ts
View file

@ -375,7 +375,7 @@ export async function runDemoTour(
yes: false,
agents: true,
scope: 'project',
mode: 'cli',
mode: 'mcp-cli',
skipAgents: false,
},
io,

13
packages/cli/src/setup-prompts.ts
View file

@ -138,9 +138,13 @@ export function createKtxSetupPromptAdapter(options: KtxSetupPromptAdapterOption
};
}
interface KtxSetupNoteOptions {
format?: (line: string) => string;
}
export interface KtxSetupUiAdapter {
intro(title: string, io: KtxCliIo): void;
note(message: string, title: string, io: KtxCliIo): void;
note(message: string, title: string, io: KtxCliIo, options?: KtxSetupNoteOptions): void;
}
function isWritableTtyOutput(output: KtxCliIo['stdout']): output is KtxCliIo['stdout'] & Writable {
@ -160,9 +164,12 @@ export function createKtxSetupUiAdapter(): KtxSetupUiAdapter {
}
io.stdout.write(`${title}\n`);
},
note(message, title, io) {
note(message, title, io, options) {
if (isWritableTtyOutput(io.stdout)) {
note(message, title, { output: io.stdout });
note(message, title, {
output: io.stdout,
...(options?.format ? { format: options.format } : {}),
});
return;
}
io.stdout.write(`\n${title}:\n`);

21
packages/cli/src/setup.test.ts
View file

@ -232,7 +232,10 @@ describe('setup status', () => {
version: 1,
projectDir: tempDir,
installedAt: '2026-05-07T00:00:00.000Z',
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }],
installs: [
{ target: 'codex', scope: 'project', mode: 'mcp' },
{ target: 'codex', scope: 'project', mode: 'mcp-cli' },
],
entries: [],
},
null,
@ -1514,7 +1517,7 @@ describe('setup status', () => {
return {
status: 'ready',
projectDir: tempDir,
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }],
installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
};
},
},
@ -1566,7 +1569,7 @@ describe('setup status', () => {
agents: async () => ({
status: 'ready',
projectDir: tempDir,
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }],
installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
}),
},
),
@ -1617,7 +1620,7 @@ describe('setup status', () => {
return {
status: 'ready',
projectDir: tempDir,
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }],
installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
};
},
},
@ -1632,7 +1635,7 @@ describe('setup status', () => {
const agents = vi.fn(async () => ({
status: 'ready' as const,
projectDir: tempDir,
installs: [{ target: 'codex' as const, scope: 'project' as const, mode: 'cli' as const }],
installs: [{ target: 'codex' as const, scope: 'project' as const, mode: 'mcp-cli' as const }],
}));
await writeFile(join(tempDir, 'ktx.yaml'), ['connections: {}', ''].join('\n'), 'utf-8');
@ -1701,7 +1704,7 @@ describe('setup status', () => {
version: 1,
projectDir: tempDir,
installedAt: '2026-05-07T00:00:00.000Z',
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }],
installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
entries: [],
},
null,
@ -1764,7 +1767,7 @@ describe('setup status', () => {
return {
status: 'ready',
projectDir: tempDir,
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }],
installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
};
},
},
@ -1856,7 +1859,7 @@ describe('setup status', () => {
return {
status: 'ready',
projectDir: tempDir,
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }],
installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
};
},
},
@ -1873,7 +1876,7 @@ describe('setup status', () => {
const agents = vi.fn(async () => ({
status: 'ready' as const,
projectDir: tempDir,
installs: [{ target: 'universal' as const, scope: 'project' as const, mode: 'cli' as const }],
installs: [{ target: 'universal' as const, scope: 'project' as const, mode: 'mcp-cli' as const }],
}));
await expect(

38
packages/cli/src/setup.ts
View file

@ -306,12 +306,15 @@ export async function readKtxSetupStatus(projectDir: string): Promise<KtxSetupSt
const databaseIds = project.config.setup?.database_connection_ids ?? Object.keys(project.config.connections);
const databasesComplete = completedSteps.includes('databases');
const manifest = await readKtxAgentInstallManifest(resolvedProjectDir);
const agents =
manifest?.installs.map((install) => ({
const agentMap = new Map<string, { target: string; scope: string; ready: boolean }>();
for (const install of manifest?.installs ?? []) {
agentMap.set(`${install.target}:${install.scope}`, {
target: install.target,
scope: install.scope,
ready: true,
})) ?? [];
});
}
const agents = [...agentMap.values()];
return {
project: { path: resolvedProjectDir, ready: true, name: basename(project.projectDir) || project.projectDir },
@ -657,7 +660,7 @@ async function runKtxSetupInner(args: KtxSetupArgs, io: KtxCliIo, deps: KtxSetup
agents: true,
...(args.target ? { target: args.target } : {}),
scope: args.agentScope ?? 'project',
mode: 'cli',
mode: 'mcp',
skipAgents: false,
},
io,
@ -702,16 +705,21 @@ async function runKtxSetupInner(args: KtxSetupArgs, io: KtxCliIo, deps: KtxSetup
await commitSetupConfigChanges(projectResult.projectDir);
const status = await readKtxSetupStatus(projectResult.projectDir);
io.stdout.write(formatKtxSetupStatus(status));
setupUi.note(
formatSetupNextStepLines({
setupReady: setupStatusReady(status),
hasContextTargets: setupHasContextTargets(status),
contextReady: setupContextReady(status),
agentIntegrationReady: status.agents.some((agent) => agent.ready),
}).join('\n'),
'What you can do next',
io,
);
const focusedOnAgents = args.agents || entryAction === 'agents';
if (!focusedOnAgents) {
setupUi.note(formatKtxSetupStatus(status).trimEnd(), 'Project status', io, {
format: (line) => line,
});
setupUi.note(
formatSetupNextStepLines({
setupReady: setupStatusReady(status),
hasContextTargets: setupHasContextTargets(status),
contextReady: setupContextReady(status),
agentIntegrationReady: status.agents.some((agent) => agent.ready),
}).join('\n'),
'What you can do next',
io,
);
}
return 0;
}

62
packages/cli/src/skills/analytics/SKILL.md Normal file
View file

@ -0,0 +1,62 @@
---
name: ktx-analytics
description: Use when answering a question that needs data from a KTX-connected database - investigating, analyzing, "how many", "show me", "what's the breakdown of", finding records by value, exploring tables, comparing periods, explaining metrics, or any data-analysis request. Triggers even when the user does not say "analytics"; if the answer requires querying a configured KTX connection, this skill applies.
---
# KTX Analytics Workflow
You have access to KTX MCP tools for data discovery, semantic-layer analysis, raw read-only SQL, wiki context, and memory ingest. Follow this workflow.
<workflow>
1. **Discover** - call `discover_data` first to see what exists across wiki pages, semantic-layer sources, metrics, dimensions, raw tables, and columns. Returns refs only.
2. **Inspect top hits in parallel** - for each promising ref:
- `kind: 'wiki'` -> `wiki_read`
- `kind: 'sl_source'`, `kind: 'sl_measure'`, or `kind: 'sl_dimension'` -> `sl_read_source`
- `kind: 'table'` or `kind: 'column'` -> `entity_details`
3. **Resolve business values** - if the user named a value such as "Acme Corp", "enterprise", or "status=shipped", call `dictionary_search` to find which column holds it.
4. **Plan the analysis** - identify the grain, metrics, dimensions, filters, time window, and expected row limits before querying.
5. **Query** -
- Prefer `sl_query` when the semantic layer covers the question.
- Use `sql_execution` only for questions the semantic layer does not cover.
6. **Validate and explain** - sanity-check totals, filters, null handling, and time zones. State the source tables or semantic-layer objects used.
7. **Capture durable learnings** - call `memory_ingest` whenever a turn produces something worth remembering (business rules, metric definitions, schema gotchas, recurring findings) **or** whenever the user asks you to remember something. Pass markdown in `content` including any source context the memory agent should weigh. Each call is a feedback loop; better notes today mean smarter `discover_data` and `wiki_search` results tomorrow.
</workflow>
<rules>
- Always run `discover_data` before writing SQL. Do not guess table names.
- Prefer the semantic layer over raw SQL when both can answer the question; measures are the source of truth.
- Read entity details before writing SQL against an unfamiliar table. Do not assume column names.
- Treat `sql_execution` as read-only. Writes are rejected by the server.
- Validate value mentions with `dictionary_search` instead of guessing case or spelling. Treat a `dictionary_search` miss as non-authoritative. The index is built from profile-sampled values, so a missing value may simply have been outside the sample. Follow up with `sql_execution` against the most plausible columns before concluding the value is absent.
- When `connection_list` shows multiple connections, pass an explicit `connectionId` to every tool that takes one and where user intent pins a specific warehouse. Required: `entity_details`, `sl_read_source`, and `sql_execution`. Required when user intent is warehouse-specific, including wording like "in our warehouse" or "this warehouse": `memory_ingest`; without `connectionId`, the memory agent cannot update the semantic layer and the knowledge lands as wiki-only. Pass `connectionId` when intent pins a warehouse, otherwise omit for unscoped discovery: `sl_query`, `discover_data`, and `dictionary_search`. Never pass `connectionId` to `connection_list`, `wiki_search`, `wiki_read`, or `memory_ingest_status`. If intent is ambiguous for a required-or-scoped tool, ask the user which warehouse before calling.
- Show compact result tables for small outputs. For broad results, summarize the top findings and mention the applied limit.
- Ask a concise clarification only when the metric, date range, entity, or grain is genuinely ambiguous and cannot be inferred from context.
</rules>
<examples>
**Input:** "How many orders did Acme Corp place last month?"
**Workflow:**
1. `dictionary_search({ values: ["Acme Corp"] })` finds `customers.name`.
2. `discover_data({ query: "orders customer monthly" })` finds an orders semantic-layer source.
3. `sl_read_source({ connectionId: "warehouse", sourceName: "orders_facts" })` confirms the source grain, measures, and dimensions.
4. `sl_query({ connectionId: "warehouse", measures: ["order_count"], filters: ["customer_name = 'Acme Corp'"] })` answers through the semantic layer.
5. `memory_ingest({ connectionId: "warehouse", content: "Acme Corp order analysis used orders_facts.order_count filtered by customers.name = 'Acme Corp'. Source: current analysis turn." })` captures the durable finding.
---
**Input:** "What columns does the events table have?"
**Workflow:**
1. `discover_data({ query: "events table" })` returns a `table` ref.
2. `entity_details({ connectionId: "warehouse", entities: [{ table: "analytics.events" }] })` returns columns, types, and foreign keys.
3. Answer directly. No query is needed.
---
**Input:** "Heads up: ARR is always reported in cents in our warehouse."
**Workflow:**
1. If multiple connections exist, call `connection_list` and identify the warehouse the user means. Ask if ambiguous.
2. `memory_ingest({ connectionId: "warehouse", content: "ARR is reported in cents (not dollars) in this warehouse. Multiply by 0.01 for dollar amounts. Source: user clarification." })` remembers the warehouse-specific rule without running an analysis turn.
</examples>

49
packages/cli/src/skills/research/SKILL.md
View file

@ -1,49 +0,0 @@
---
name: ktx-research
description: Use when answering a question that needs data from a KTX-connected database - investigating, analyzing, "how many", "show me", "what's the breakdown of", finding records by value, exploring tables, comparing periods, or any data-investigation request. Triggers even when the user does not say "research"; if the answer requires querying a configured KTX connection, this skill applies.
---
# KTX Research Workflow
You have access to KTX MCP tools for investigating data. Follow this workflow.
<workflow>
1. **Discover** - call `discover_data` first to see what exists across wiki, semantic-layer sources, and raw tables. Returns refs only.
2. **Inspect top hits in parallel** - for each promising ref:
- `kind: 'wiki'` -> `wiki_read`
- `kind: 'sl_source'`, `kind: 'sl_measure'`, or `kind: 'sl_dimension'` -> `sl_read_source`
- `kind: 'table'` or `kind: 'column'` -> `entity_details`
3. **Resolve literals** - if the user named a value such as "Acme Corp" or "status=shipped", call `dictionary_search` to find which column holds it.
4. **Query** -
- Prefer `sl_query` when the semantic layer covers the question.
- Use `sql_execution` only for questions the semantic layer does not cover.
5. **Capture learnings** - at the end of the turn, call `memory_capture` so future turns benefit. Skip when the answer carries no durable knowledge.
</workflow>
<rules>
- Always run `discover_data` before writing SQL. Do not guess table names.
- Prefer the semantic layer over raw SQL when both can answer the question; measures are the source of truth.
- Read entity details before writing SQL against an unfamiliar table. Do not assume column names.
- Treat `sql_execution` as read-only. Writes are rejected by the server.
- Validate value mentions with `dictionary_search` instead of guessing case or spelling. Treat a `dictionary_search` miss as non-authoritative. The index is built from profile-sampled values, so a missing value may simply have been outside the sample. Follow up with `sql_execution` against the most plausible columns before concluding the value is absent.
</rules>
<examples>
**Input:** "How many orders did Acme Corp place last month?"
**Workflow:**
1. `dictionary_search({ values: ["Acme Corp"] })` finds `customers.name`.
2. `discover_data({ query: "orders customer monthly" })` finds an orders semantic-layer source.
3. `sl_read_source({ connectionId: "warehouse", sourceName: "orders_facts" })` confirms the source grain, measures, and dimensions.
4. `sl_query({ connectionId: "warehouse", measures: ["order_count"], filters: ["customer_name = 'Acme Corp'"] })` answers through the semantic layer.
5. `memory_capture({ userMessage, assistantMessage })` captures the durable finding.
---
**Input:** "What columns does the events table have?"
**Workflow:**
1. `discover_data({ query: "events table" })` returns a `table` ref.
2. `entity_details({ connectionId: "warehouse", entities: [{ table: "analytics.events" }] })` returns columns, types, and foreign keys.
3. Answer directly. No query is needed.
</examples>

74
packages/cli/src/text-ingest.test.ts
View file

@ -1,7 +1,7 @@
import { describe, expect, it, vi } from 'vitest';
import type { MemoryCaptureStatus } from '@ktx/context/memory';
import type { MemoryIngestStatus } from '@ktx/context/memory';
import type { KtxLocalProject } from '@ktx/context/project';
import { runKtxTextIngest, type TextMemoryCapturePort } from './text-ingest.js';
import { runKtxTextIngest, type TextMemoryIngestPort } from './text-ingest.js';
function makeIo(options: { isTTY?: boolean } = {}) {
let stdout = '';
@ -25,18 +25,18 @@ function makeIo(options: { isTTY?: boolean } = {}) {
};
}
function fakeCapture(
function fakeIngest(
options: {
failRunIds?: Set<string>;
missingStatusRunIds?: Set<string>;
events?: string[];
} = {},
): TextMemoryCapturePort {
): TextMemoryIngestPort {
let next = 1;
return {
capture: vi.fn(async () => {
ingest: vi.fn(async () => {
const runId = `run-${next++}`;
options.events?.push(`capture:${runId}`);
options.events?.push(`ingest:${runId}`);
return { runId };
}),
waitForRun: vi.fn(async (runId: string) => {
@ -51,26 +51,26 @@ function fakeCapture(
return {
runId,
status: 'error',
stage: 'capturing',
stage: 'ingesting',
done: true,
captured: { wiki: [], sl: [], xrefs: [] },
error: `${runId} failed`,
commitHash: null,
skillsLoaded: [],
signalDetected: false,
} satisfies MemoryCaptureStatus;
} satisfies MemoryIngestStatus;
}
return {
runId,
status: 'done',
stage: 'capturing',
stage: 'ingesting',
done: true,
captured: { wiki: [`wiki-${runId}`], sl: [`sl-${runId}`], xrefs: [] },
error: null,
commitHash: `commit-${runId}`,
skillsLoaded: ['wiki_capture', 'sl'],
signalDetected: true,
} satisfies MemoryCaptureStatus;
} satisfies MemoryIngestStatus;
}),
};
}
@ -80,11 +80,11 @@ function fakeProject(projectDir = '/tmp/project'): KtxLocalProject {
}
describe('runKtxTextIngest', () => {
it('captures repeated inline text sequentially with generated internal chat ids', async () => {
it('ingests repeated inline text sequentially with generated internal chat ids', async () => {
const io = makeIo();
const events: string[] = [];
const capture = fakeCapture({ events });
const createMemoryCapture = vi.fn(() => capture);
const ingest = fakeIngest({ events });
const createMemoryIngest = vi.fn(() => ingest);
await expect(
runKtxTextIngest(
@ -99,14 +99,14 @@ describe('runKtxTextIngest', () => {
io.io,
{
loadProject: vi.fn(async () => fakeProject()),
createMemoryCapture,
createMemoryIngest,
now: () => 1_700_000_000_000,
},
),
).resolves.toBe(0);
expect(createMemoryCapture).toHaveBeenCalledWith({ projectDir: '/tmp/project' });
expect(capture.capture).toHaveBeenNthCalledWith(
expect(createMemoryIngest).toHaveBeenCalledWith({ projectDir: '/tmp/project' });
expect(ingest.ingest).toHaveBeenNthCalledWith(
1,
expect.objectContaining({
userId: 'local-cli',
@ -116,7 +116,7 @@ describe('runKtxTextIngest', () => {
sourceType: 'external_ingest',
}),
);
expect(capture.capture).toHaveBeenNthCalledWith(
expect(ingest.ingest).toHaveBeenNthCalledWith(
2,
expect.objectContaining({
chatId: 'cli-text-ingest-1700000000000-2',
@ -124,8 +124,8 @@ describe('runKtxTextIngest', () => {
assistantMessage: 'Orders are completed purchases.',
}),
);
expect(capture.capture).not.toHaveBeenCalledWith(expect.objectContaining({ connectionId: expect.anything() }));
expect(events).toEqual(['capture:run-1', 'wait:run-1', 'status:run-1', 'capture:run-2', 'wait:run-2', 'status:run-2']);
expect(ingest.ingest).not.toHaveBeenCalledWith(expect.objectContaining({ connectionId: expect.anything() }));
expect(events).toEqual(['ingest:run-1', 'wait:run-1', 'status:run-1', 'ingest:run-2', 'wait:run-2', 'status:run-2']);
expect(JSON.parse(io.stdout())).toMatchObject({
status: 'done',
results: [
@ -147,7 +147,7 @@ describe('runKtxTextIngest', () => {
it('loads files and stdin as batch items and passes a global connection id', async () => {
const io = makeIo();
const capture = fakeCapture();
const ingest = fakeIngest();
await expect(
runKtxTextIngest(
@ -163,7 +163,7 @@ describe('runKtxTextIngest', () => {
io.io,
{
loadProject: vi.fn(async () => fakeProject()),
createMemoryCapture: vi.fn(() => capture),
createMemoryIngest: vi.fn(() => ingest),
readFile: vi.fn(async (path) => `file:${path}`),
readStdin: vi.fn(async () => 'stdin content'),
now: () => 10,
@ -171,7 +171,7 @@ describe('runKtxTextIngest', () => {
),
).resolves.toBe(0);
expect(capture.capture).toHaveBeenNthCalledWith(
expect(ingest.ingest).toHaveBeenNthCalledWith(
1,
expect.objectContaining({
connectionId: 'warehouse',
@ -180,7 +180,7 @@ describe('runKtxTextIngest', () => {
assistantMessage: 'file:/tmp/docs/revenue.md',
}),
);
expect(capture.capture).toHaveBeenNthCalledWith(
expect(ingest.ingest).toHaveBeenNthCalledWith(
2,
expect.objectContaining({
connectionId: 'warehouse',
@ -194,9 +194,9 @@ describe('runKtxTextIngest', () => {
expect(io.stdout()).toContain('stdin');
});
it('uses bounded inline text previews as labels in plain output and capture metadata', async () => {
it('uses bounded inline text previews as labels in plain output and ingest metadata', async () => {
const io = makeIo();
const capture = fakeCapture();
const ingest = fakeIngest();
const longText = `This inline note is intentionally long ${'x'.repeat(120)}`;
await expect(
@ -212,7 +212,7 @@ describe('runKtxTextIngest', () => {
io.io,
{
loadProject: vi.fn(async () => fakeProject()),
createMemoryCapture: vi.fn(() => capture),
createMemoryIngest: vi.fn(() => ingest),
now: () => 10,
},
),
@ -225,19 +225,19 @@ describe('runKtxTextIngest', () => {
expect(output).not.toContain('text-1');
expect(output).not.toContain(longText);
expect(capture.capture).toHaveBeenNthCalledWith(
expect(ingest.ingest).toHaveBeenNthCalledWith(
1,
expect.objectContaining({
userMessage: 'Ingest external text artifact "remember to call me Andrey" into KTX memory.',
}),
);
expect(capture.capture).toHaveBeenNthCalledWith(
expect(ingest.ingest).toHaveBeenNthCalledWith(
2,
expect.objectContaining({
userMessage: 'Ingest external text artifact "first line second line" into KTX memory.',
}),
);
expect(capture.capture).toHaveBeenNthCalledWith(
expect(ingest.ingest).toHaveBeenNthCalledWith(
3,
expect.objectContaining({
userMessage: 'Ingest external text artifact "This inline note is intentionally long xxxxxxxx..." into KTX memory.',
@ -247,7 +247,7 @@ describe('runKtxTextIngest', () => {
it('continues after an item failure by default and stops when failFast is set', async () => {
const continueIo = makeIo();
const continueCapture = fakeCapture({ failRunIds: new Set(['run-1']) });
const continueIngest = fakeIngest({ failRunIds: new Set(['run-1']) });
await expect(
runKtxTextIngest(
@ -262,12 +262,12 @@ describe('runKtxTextIngest', () => {
continueIo.io,
{
loadProject: vi.fn(async () => fakeProject()),
createMemoryCapture: vi.fn(() => continueCapture),
createMemoryIngest: vi.fn(() => continueIngest),
},
),
).resolves.toBe(1);
expect(continueCapture.capture).toHaveBeenCalledTimes(2);
expect(continueIngest.ingest).toHaveBeenCalledTimes(2);
expect(JSON.parse(continueIo.stdout())).toMatchObject({
status: 'failed',
results: [
@ -277,7 +277,7 @@ describe('runKtxTextIngest', () => {
});
const failFastIo = makeIo();
const failFastCapture = fakeCapture({ failRunIds: new Set(['run-1']) });
const failFastIngest = fakeIngest({ failRunIds: new Set(['run-1']) });
await expect(
runKtxTextIngest(
@ -292,12 +292,12 @@ describe('runKtxTextIngest', () => {
failFastIo.io,
{
loadProject: vi.fn(async () => fakeProject()),
createMemoryCapture: vi.fn(() => failFastCapture),
createMemoryIngest: vi.fn(() => failFastIngest),
},
),
).resolves.toBe(1);
expect(failFastCapture.capture).toHaveBeenCalledTimes(1);
expect(failFastIngest.ingest).toHaveBeenCalledTimes(1);
expect(JSON.parse(failFastIo.stdout()).results).toHaveLength(1);
});
@ -314,7 +314,7 @@ describe('runKtxTextIngest', () => {
failFast: false,
},
noInputIo.io,
{ loadProject: vi.fn(), createMemoryCapture: vi.fn() },
{ loadProject: vi.fn(), createMemoryIngest: vi.fn() },
),
).resolves.toBe(1);
expect(noInputIo.stderr()).toContain('Provide at least one text item');
@ -331,7 +331,7 @@ describe('runKtxTextIngest', () => {
failFast: false,
},
emptyIo.io,
{ loadProject: vi.fn(), createMemoryCapture: vi.fn() },
{ loadProject: vi.fn(), createMemoryIngest: vi.fn() },
),
).resolves.toBe(1);
expect(emptyIo.stderr()).toContain('Text item "text-1" is empty');

36
packages/cli/src/text-ingest.ts
View file

@ -1,6 +1,6 @@
import { readFile as fsReadFile } from 'node:fs/promises';
import { basename, resolve } from 'node:path';
import { createLocalProjectMemoryCapture, type MemoryAgentInput, type MemoryCaptureStatus } from '@ktx/context/memory';
import { createLocalProjectMemoryIngest, type MemoryAgentInput, type MemoryIngestStatus } from '@ktx/context/memory';
import { loadKtxProject, type KtxLocalProject } from '@ktx/context/project';
import type { KtxCliIo } from './cli-runtime.js';
import { createRepainter, initViewState, renderContextBuildView, type ContextBuildTargetState } from './context-build-view.js';
@ -17,10 +17,10 @@ export interface KtxTextIngestArgs {
failFast: boolean;
}
export interface TextMemoryCapturePort {
capture(input: MemoryAgentInput): Promise<{ runId: string }>;
export interface TextMemoryIngestPort {
ingest(input: MemoryAgentInput): Promise<{ runId: string }>;
waitForRun(runId: string): Promise<void>;
status(runId: string): Promise<MemoryCaptureStatus | null>;
status(runId: string): Promise<MemoryIngestStatus | null>;
}
interface TextIngestItem {
@ -32,14 +32,14 @@ interface TextIngestResult {
label: string;
runId: string | null;
status: 'done' | 'error';
captured: MemoryCaptureStatus['captured'];
captured: MemoryIngestStatus['captured'];
commitHash: string | null;
error: string | null;
}
export interface KtxTextIngestDeps {
loadProject?: (options: { projectDir: string }) => Promise<KtxLocalProject>;
createMemoryCapture?: (project: KtxLocalProject) => TextMemoryCapturePort;
createMemoryIngest?: (project: KtxLocalProject) => TextMemoryIngestPort;
readFile?: (path: string) => Promise<string>;
readStdin?: () => Promise<string>;
now?: () => number;
@ -48,8 +48,8 @@ export interface KtxTextIngestDeps {
const INLINE_TEXT_LABEL_MAX_LENGTH = 50;
const ANSI_ESCAPE_PATTERN = /\x1B\[[0-?]*[ -/]*[@-~]/g;
function defaultCreateMemoryCapture(project: KtxLocalProject): TextMemoryCapturePort {
return createLocalProjectMemoryCapture(project);
function defaultCreateMemoryIngest(project: KtxLocalProject): TextMemoryIngestPort {
return createLocalProjectMemoryIngest(project);
}
async function defaultReadStdin(): Promise<string> {
@ -65,7 +65,7 @@ async function defaultReadFile(path: string): Promise<string> {
return await fsReadFile(path, 'utf-8');
}
function emptyCaptured(): MemoryCaptureStatus['captured'] {
function emptyCaptured(): MemoryIngestStatus['captured'] {
return { wiki: [], sl: [], xrefs: [] };
}
@ -182,7 +182,7 @@ function renderTextIngestView(state: ReturnType<typeof initViewState>, styled: b
});
}
function summarizeCaptured(captured: MemoryCaptureStatus['captured']): string {
function summarizeCaptured(captured: MemoryIngestStatus['captured']): string {
const parts = [
`wiki=${captured.wiki.length}`,
`sl=${captured.sl.length}`,
@ -191,7 +191,7 @@ function summarizeCaptured(captured: MemoryCaptureStatus['captured']): string {
return parts.join(', ');
}
function resultFromStatus(label: string, status: MemoryCaptureStatus): TextIngestResult {
function resultFromStatus(label: string, status: MemoryIngestStatus): TextIngestResult {
return {
label,
runId: status.runId,
@ -251,7 +251,7 @@ export async function runKtxTextIngest(
}
const project = await (deps.loadProject ?? loadKtxProject)({ projectDir: args.projectDir });
const memoryCapture = (deps.createMemoryCapture ?? defaultCreateMemoryCapture)(project);
const memoryIngest = (deps.createMemoryIngest ?? defaultCreateMemoryIngest)(project);
const now = deps.now ?? (() => Date.now());
const batchId = now();
const state = initViewState(items.map((item) => makeTarget(item.label)));
@ -292,7 +292,7 @@ export async function runKtxTextIngest(
let runId: string | null = null;
let result: TextIngestResult;
try {
const captureInput: MemoryAgentInput = {
const ingestInput: MemoryAgentInput = {
userId: args.userId,
chatId: `cli-text-ingest-${batchId}-${index + 1}`,
userMessage: `Ingest external text artifact ${artifactReference(item.label)} into KTX memory.`,
@ -300,12 +300,12 @@ export async function runKtxTextIngest(
...(args.connectionId ? { connectionId: args.connectionId } : {}),
sourceType: 'external_ingest',
};
const capture = await memoryCapture.capture(captureInput);
runId = capture.runId;
await memoryCapture.waitForRun(runId);
const status = await memoryCapture.status(runId);
const ingest = await memoryIngest.ingest(ingestInput);
runId = ingest.runId;
await memoryIngest.waitForRun(runId);
const status = await memoryIngest.status(runId);
if (!status) {
throw new Error(`Memory capture run "${runId}" was not found.`);
throw new Error(`Memory ingest run "${runId}" was not found.`);
}
result = resultFromStatus(item.label, status);
} catch (error) {

14
packages/context/src/daemon/semantic-layer-compute.ts
View file

@ -2,7 +2,7 @@ import { request as httpRequest } from 'node:http';
import { request as httpsRequest } from 'node:https';
import { URL } from 'node:url';
import { spawn } from 'node:child_process';
import type { SemanticLayerQueryInput, SemanticLayerSource } from '../sl/index.js';
import type { ResolvedSemanticLayerSource, SemanticLayerQueryInput } from '../sl/types.js';
export interface KtxSemanticLayerComputeQueryResult {
sql: string;
@ -54,13 +54,21 @@ export interface KtxSemanticLayerSourceGenerationResult {
}
export interface KtxSemanticLayerComputePort {
/**
* Callers must pass sources sanitized through toResolvedWire. The Python
* daemon rejects authoring-only fields such as usage and inherits_columns_from.
*/
query(input: {
sources: Array<Record<string, unknown> | SemanticLayerSource>;
sources: ResolvedSemanticLayerSource[];
query: SemanticLayerQueryInput;
dialect: string;
}): Promise<KtxSemanticLayerComputeQueryResult>;
/**
* Callers must pass sources sanitized through toResolvedWire. The Python
* daemon rejects authoring-only fields such as usage and inherits_columns_from.
*/
validateSources(input: {
sources: Array<Record<string, unknown> | SemanticLayerSource>;
sources: ResolvedSemanticLayerSource[];
dialect: string;
recentlyTouched?: string[];
}): Promise<KtxSemanticLayerComputeValidationResult>;

30
packages/context/src/llm/local-config.test.ts
View file

@ -191,6 +191,36 @@ describe('local KTX embedding config', () => {
expect(resolveLocalKtxEmbeddingConfig(config, {})).toBeNull();
});
it('returns null when backend is openai but no apiKey is resolvable from env', () => {
const config: KtxProjectEmbeddingConfig = {
backend: 'openai',
model: 'text-embedding-3-small',
dimensions: 1536,
openai: { api_key: 'env:OPENAI_API_KEY' }, // pragma: allowlist secret
};
expect(resolveLocalKtxEmbeddingConfig(config, {})).toBeNull();
});
it('resolves openai embedding config from env', () => {
const config: KtxProjectEmbeddingConfig = {
backend: 'openai',
model: 'text-embedding-3-small',
dimensions: 1536,
openai: { api_key: 'env:OPENAI_API_KEY' }, // pragma: allowlist secret
};
expect(
resolveLocalKtxEmbeddingConfig(config, { OPENAI_API_KEY: 'sk-test' }), // pragma: allowlist secret
).toEqual({
backend: 'openai',
model: 'text-embedding-3-small',
dimensions: 1536,
openai: { apiKey: 'sk-test' }, // pragma: allowlist secret
batchSize: undefined,
});
});
it('constructs deterministic embeddings from the default project config', () => {
const createKtxEmbeddingProvider = vi.fn(() => ({}) as never);
const provider = createLocalKtxEmbeddingProviderFromConfig(

14
packages/context/src/llm/local-config.ts
View file

@ -145,11 +145,23 @@ export function resolveLocalKtxEmbeddingConfig(
batchSize: config.batchSize,
};
}
if (config.backend === 'openai') {
const openai = resolvedProviderConfig(config.openai, env);
if (!openai?.apiKey) {
return null;
}
return {
backend: config.backend,
model: config.model ?? 'deterministic',
dimensions: config.dimensions,
openai,
batchSize: config.batchSize,
};
}
return {
backend: config.backend,
model: config.model ?? 'deterministic',
dimensions: config.dimensions,
...(resolvedProviderConfig(config.openai, env) ? { openai: resolvedProviderConfig(config.openai, env) } : {}),
...(config.sentenceTransformers
? {
sentenceTransformers: {

1615
packages/context/src/mcp/__snapshots__/mcp-tools-list.json Normal file
View file

File diff suppressed because it is too large Load diff

925
packages/context/src/mcp/context-tools.ts
View file

File diff suppressed because it is too large Load diff

13
packages/context/src/mcp/index.ts
View file

@ -8,29 +8,18 @@ export type {
KtxDiscoverDataMcpPort,
KtxDictionarySearchMcpPort,
KtxEntityDetailsMcpPort,
KtxIngestDiffSummary,
KtxIngestMcpPort,
KtxIngestStatusResponse,
KtxIngestTriggerKind,
KtxIngestTriggerResponse,
KtxIngestWorkUnitSummary,
KtxKnowledgeMcpPort,
KtxKnowledgePage,
KtxKnowledgeSearchResponse,
KtxKnowledgeSearchResult,
KtxKnowledgeWriteResponse,
KtxMcpContextPorts,
KtxMcpServerDeps,
KtxMcpServerLike,
KtxMcpTextContent,
KtxMcpToolResult,
KtxMcpUserContext,
KtxSemanticLayerListResponse,
KtxSemanticLayerMcpPort,
KtxSemanticLayerQueryResponse,
KtxSemanticLayerReadResponse,
KtxSemanticLayerSourceSummary,
KtxSemanticLayerValidationResponse,
KtxSemanticLayerWriteResponse,
MemoryCapturePort,
MemoryIngestPort,
} from './types.js';

1049
packages/context/src/mcp/local-project-ports.test.ts
View file

File diff suppressed because it is too large Load diff

556
packages/context/src/mcp/local-project-ports.ts
View file

@ -1,65 +1,19 @@
import YAML from 'yaml';
import {
type KtxSqlQueryExecutorPort,
localConnectionInfoFromConfig,
localConnectionTypeForConfig,
} from '../connections/index.js';
import { type KtxSqlQueryExecutorPort, localConnectionInfoFromConfig } from '../connections/index.js';
import type { KtxEmbeddingPort } from '../core/index.js';
import type { KtxSemanticLayerComputePort } from '../daemon/index.js';
import {
createDefaultLocalIngestAdapters,
getLocalIngestStatus,
type IngestReportSnapshot,
ingestReportToMemoryFlowReplay,
type LocalIngestMcpOptions,
runLocalIngest,
runLocalMetabaseIngest,
} from '../ingest/index.js';
import { createLocalKtxEmbeddingProviderFromConfig, KtxIngestEmbeddingPortAdapter } from '../llm/index.js';
import type { KtxLocalProject } from '../project/index.js';
import {
createKtxEntityDetailsService,
getLocalScanReport,
getLocalScanStatus,
type KtxConnectionDriver,
type KtxScanConnector,
type KtxScanReport,
type LocalScanMcpOptions,
runLocalScan,
} from '../scan/index.js';
import { createKtxEntityDetailsService, type KtxScanConnector, type LocalScanMcpOptions } from '../scan/index.js';
import { createKtxDiscoverDataService } from '../search/index.js';
import type { SqlAnalysisDialect, SqlAnalysisPort } from '../sql-analysis/index.js';
import {
compileLocalSlQuery,
createKtxDictionarySearchService,
type LocalSlSourceSearchResult,
type LocalSlSourceSummary,
listLocalSlSources,
searchLocalSlSources,
sourceDefinitionSchema,
sourceOverlaySchema,
} from '../sl/index.js';
import { readLocalKnowledgePage, searchLocalKnowledgePages, writeLocalKnowledgePage } from '../wiki/local-knowledge.js';
import type {
KtxConnectionTestResponse,
KtxIngestStatusResponse,
KtxMcpContextPorts,
KtxScanArtifactListResponse,
KtxScanArtifactReadResponse,
KtxScanArtifactSummary,
KtxScanArtifactType,
KtxSqlExecutionResponse,
} from './types.js';
const LOCAL_AUTHOR = 'ktx';
const LOCAL_AUTHOR_EMAIL = 'ktx@example.com';
const SL_SHAPE_WARNING = 'Local stdio validation checks YAML shape only; Python semantic validation is not configured.';
import { compileLocalSlQuery, createKtxDictionarySearchService } from '../sl/index.js';
import { readLocalKnowledgePage, searchLocalKnowledgePages } from '../wiki/local-knowledge.js';
import type { KtxMcpContextPorts, KtxMcpProgressCallback, KtxSqlExecutionResponse } from './types.js';
interface CreateLocalProjectMcpContextPortsOptions {
semanticLayerCompute?: KtxSemanticLayerComputePort;
queryExecutor?: KtxSqlQueryExecutorPort;
sqlAnalysis?: SqlAnalysisPort;
localIngest?: LocalIngestMcpOptions;
localScan?: LocalScanMcpOptions;
embeddingService?: KtxEmbeddingPort | null;
}
@ -115,284 +69,23 @@ function assertSafeSourceName(sourceName: string): string {
return assertSafePathToken('semantic-layer source name', sourceName);
}
function normalizeScanDriver(driver: string | undefined): KtxConnectionDriver {
const normalized = (driver ?? '').toLowerCase();
if (
normalized === 'postgres' ||
normalized === 'postgresql' ||
normalized === 'sqlite' ||
normalized === 'sqlite3' ||
normalized === 'mysql' ||
normalized === 'clickhouse' ||
normalized === 'sqlserver' ||
normalized === 'bigquery' ||
normalized === 'snowflake'
) {
return normalized === 'sqlite3' ? 'sqlite' : normalized;
}
return 'postgres';
}
async function cleanupConnector(connector: KtxScanConnector | null): Promise<void> {
if (connector?.cleanup) {
await connector.cleanup();
}
}
async function testLocalConnection(
project: KtxLocalProject,
options: CreateLocalProjectMcpContextPortsOptions,
connectionId: string,
): Promise<KtxConnectionTestResponse | null> {
const safeConnectionId = assertSafeConnectionId(connectionId);
const connection = project.config.connections[safeConnectionId];
if (!connection) {
return null;
}
const connectionType = localConnectionTypeForConfig(safeConnectionId, connection);
const createConnector = options.localScan?.createConnector;
if (!createConnector) {
return {
id: safeConnectionId,
connectionType,
ok: true,
tableCount: null,
message: 'Connection is configured; no native scan connector is available for live testing.',
warnings: ['ktx serve was not configured with a local scan connector factory.'],
};
}
let connector: KtxScanConnector | null = null;
try {
connector = await createConnector(safeConnectionId);
const snapshot = await connector.introspect(
{
connectionId: safeConnectionId,
driver: normalizeScanDriver(connection.driver),
mode: 'structural',
dryRun: true,
detectRelationships: false,
},
{ runId: `connection-test-${safeConnectionId}` },
);
return {
id: safeConnectionId,
connectionType,
ok: true,
tableCount: snapshot.tables.length,
message: 'Connection test passed.',
warnings: [],
};
} catch (error) {
return {
id: safeConnectionId,
connectionType,
ok: false,
tableCount: null,
message: error instanceof Error ? error.message : String(error),
warnings: [],
};
} finally {
await cleanupConnector(connector);
}
}
function scanArtifactType(path: string, report: KtxScanReport): KtxScanArtifactType {
if (path === report.artifactPaths.reportPath) {
return 'report';
}
if (report.artifactPaths.manifestShards.includes(path)) {
return 'manifest_shard';
}
if (report.artifactPaths.enrichmentArtifacts.includes(path)) {
return 'enrichment_artifact';
}
return 'raw_source';
}
async function artifactSize(project: KtxLocalProject, path: string): Promise<number | undefined> {
try {
const result = await project.fileStore.readFile(path);
return typeof result.size === 'number' ? result.size : undefined;
} catch {
return undefined;
}
}
async function listArtifactsForReport(
project: KtxLocalProject,
runId: string,
report: KtxScanReport,
): Promise<KtxScanArtifactListResponse> {
const paths = new Set<string>();
if (report.artifactPaths.rawSourcesDir) {
const listed = await project.fileStore.listFiles(report.artifactPaths.rawSourcesDir);
for (const file of listed.files) {
paths.add(file);
}
}
if (report.artifactPaths.reportPath) {
paths.add(report.artifactPaths.reportPath);
}
for (const path of report.artifactPaths.manifestShards) {
paths.add(path);
}
for (const path of report.artifactPaths.enrichmentArtifacts) {
paths.add(path);
}
const artifacts: KtxScanArtifactSummary[] = [];
for (const path of [...paths].sort()) {
const size = await artifactSize(project, path);
artifacts.push({
path,
type: scanArtifactType(path, report),
...(size === undefined ? {} : { size }),
});
}
return { runId, artifacts };
}
async function readScanArtifact(
project: KtxLocalProject,
runId: string,
path: string,
): Promise<KtxScanArtifactReadResponse | null> {
const report = await getLocalScanReport(project, runId);
if (!report) {
return null;
}
const listed = await listArtifactsForReport(project, runId, report);
const artifact = listed.artifacts.find((candidate) => candidate.path === path);
if (!artifact) {
return null;
}
const result = await project.fileStore.readFile(path);
return {
runId,
path,
type: artifact.type,
...(typeof result.size === 'number' ? { size: result.size } : {}),
content: result.content,
};
}
function slPath(connectionId: string, sourceName: string): string {
return `semantic-layer/${assertSafeConnectionId(connectionId)}/${assertSafeSourceName(sourceName)}.yaml`;
}
function sourceNameFromPath(path: string): string {
return (
path
.split('/')
.at(-1)
?.replace(/\.ya?ml$/, '') ?? path
);
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
function parseYamlRecord(raw: string): Record<string, unknown> {
const parsed = YAML.parse(raw) as unknown;
if (!isRecord(parsed)) {
throw new Error('Semantic-layer source YAML must contain an object');
}
return parsed;
}
async function listSlPaths(project: KtxLocalProject, connectionId?: string): Promise<string[]> {
const root = connectionId ? `semantic-layer/${assertSafeConnectionId(connectionId)}` : 'semantic-layer';
const listed = await project.fileStore.listFiles(root);
return listed.files.filter((file) => file.endsWith('.yaml') || file.endsWith('.yml')).sort();
}
async function loadComputableSources(
project: KtxLocalProject,
connectionId: string,
): Promise<Record<string, unknown>[]> {
const paths = await listSlPaths(project, connectionId);
const sources: Record<string, unknown>[] = [];
for (const path of paths) {
const raw = await project.fileStore.readFile(path);
const source = parseYamlRecord(raw.content);
if (source.table || source.sql) {
sources.push(source);
}
}
return sources;
}
function validateSourceRecord(sourceName: string, source: Record<string, unknown>): string[] {
const namedSource = { ...source, name: typeof source.name === 'string' ? source.name : sourceName };
const definition = sourceDefinitionSchema.safeParse(namedSource);
if (definition.success) {
return [];
}
const overlay = sourceOverlaySchema.safeParse(namedSource);
if (overlay.success) {
return [];
}
return definition.error.issues.map((issue) => `${sourceName}: ${issue.path.join('.') || 'source'} ${issue.message}`);
}
function localIngestSourceDir(config: unknown): string | undefined {
if (!isRecord(config) || config.sourceDir === undefined) {
return undefined;
}
if (typeof config.sourceDir !== 'string' || config.sourceDir.trim().length === 0) {
throw new Error('Local ingest config sourceDir must be a non-empty string when provided');
}
return config.sourceDir;
}
function rawFileCountFromIngestReport(report: IngestReportSnapshot): number {
return new Set(report.body.workUnits.flatMap((workUnit) => workUnit.rawFiles)).size;
}
function hasSlSearchMetadata(
source: LocalSlSourceSummary | LocalSlSourceSearchResult,
): source is LocalSlSourceSearchResult {
return 'score' in source;
}
function statusFromIngestReport(report: IngestReportSnapshot): KtxIngestStatusResponse {
const failedWorkUnits = report.body.failedWorkUnits;
return {
runId: report.runId,
jobId: report.jobId,
reportId: report.id,
status: failedWorkUnits.length > 0 ? 'error' : 'done',
stage: 'done',
progress: 1,
errors: failedWorkUnits,
done: true,
adapter: report.sourceKey,
connectionId: report.connectionId,
sourceDir: null,
syncId: report.body.syncId,
startedAt: report.createdAt,
completedAt: report.createdAt,
previousRunId: null,
diffSummary: report.body.diffSummary,
workUnitCount: report.body.workUnits.length,
rawFileCount: rawFileCountFromIngestReport(report),
workUnits: report.body.workUnits.map((workUnit) => ({
unitKey: workUnit.unitKey,
rawFiles: [...workUnit.rawFiles],
peerFileIndex: [],
dependencyPaths: [],
})),
evictionDeletedRawPaths: [...report.body.evictionInputs],
};
}
async function executeValidatedReadOnlySql(
project: KtxLocalProject,
options: CreateLocalProjectMcpContextPortsOptions,
input: { connectionId: string; sql: string; maxRows: number },
onProgress?: KtxMcpProgressCallback,
): Promise<KtxSqlExecutionResponse> {
await onProgress?.({ progress: 0, message: 'Validating SQL' });
const connectionId = assertSafeConnectionId(input.connectionId);
const connection = project.config.connections[connectionId];
if (!connection) {
@ -416,6 +109,7 @@ async function executeValidatedReadOnlySql(
if (!connector.capabilities.readOnlySql || !connector.executeReadOnly) {
throw new Error(`Connection "${connectionId}" does not support read-only SQL execution.`);
}
await onProgress?.({ progress: 0.3, message: 'Executing' });
const result = await connector.executeReadOnly(
{
connectionId,
@ -424,12 +118,14 @@ async function executeValidatedReadOnlySql(
},
{ runId: 'mcp-sql-execution' },
);
return {
const response = {
headers: result.headers,
...(result.headerTypes ? { headerTypes: result.headerTypes } : {}),
rows: result.rows,
rowCount: result.rowCount ?? result.rows.length,
};
await onProgress?.({ progress: 1, message: `Fetched ${response.rowCount} rows` });
return response;
} finally {
await cleanupConnector(connector);
}
@ -453,9 +149,6 @@ export function createLocalProjectMcpContextPorts(
)
.sort((a, b) => a.id.localeCompare(b.id));
},
async test(input) {
return testLocalConnection(project, options, input.connectionId);
},
},
knowledge: {
async search(input) {
@ -495,58 +188,8 @@ export function createLocalProjectMcpContextPorts(
}
: null;
},
async write(input) {
const existing = await readLocalKnowledgePage(project, {
key: input.key,
userId: input.userId,
});
await writeLocalKnowledgePage(project, {
key: input.key,
scope: 'GLOBAL',
userId: input.userId,
summary: input.summary,
content: input.content,
tags: input.tags,
refs: input.refs,
slRefs: input.slRefs,
source: input.source,
intent: input.intent,
tables: input.tables,
representativeSql: input.representativeSql,
usage: input.usage,
fingerprints: input.fingerprints,
});
return { success: true, key: input.key, action: existing ? 'updated' : 'created' };
},
},
semanticLayer: {
async listSources(input) {
const listed: Array<LocalSlSourceSummary | LocalSlSourceSearchResult> = input.query
? await searchLocalSlSources(project, {
connectionId: input.connectionId,
query: input.query,
embeddingService,
})
: await listLocalSlSources(project, { connectionId: input.connectionId });
const sources = listed.map((source) => ({
connectionId: source.connectionId,
connectionName: source.connectionId,
name: source.name,
description: source.description,
columnCount: source.columnCount,
measureCount: source.measureCount,
joinCount: source.joinCount,
...(hasSlSearchMetadata(source) && source.frequencyTier ? { frequencyTier: source.frequencyTier } : {}),
...(hasSlSearchMetadata(source) && source.snippet ? { snippet: source.snippet } : {}),
...(hasSlSearchMetadata(source) ? { score: source.score } : {}),
...(hasSlSearchMetadata(source) && source.matchReasons ? { matchReasons: source.matchReasons } : {}),
...(hasSlSearchMetadata(source) && source.dictionaryMatches
? { dictionaryMatches: source.dictionaryMatches }
: {}),
...(hasSlSearchMetadata(source) && source.lanes ? { lanes: source.lanes } : {}),
}));
return { sources, totalSources: sources.length };
},
async readSource(input) {
const path = slPath(input.connectionId, input.sourceName);
try {
@ -556,71 +199,9 @@ export function createLocalProjectMcpContextPorts(
return null;
}
},
async writeSource(input) {
const path = slPath(input.connectionId, input.sourceName);
if (input.delete) {
const deleted = await project.fileStore.deleteFile(
path,
LOCAL_AUTHOR,
LOCAL_AUTHOR_EMAIL,
`Remove semantic-layer source: ${input.sourceName}`,
);
return { success: Boolean(deleted), sourceName: input.sourceName };
}
const yaml =
input.yaml ?? YAML.stringify({ ...input.source, name: input.sourceName }, { indent: 2, lineWidth: 0, version: '1.1' });
parseYamlRecord(yaml);
await project.fileStore.writeFile(
path,
`${yaml.trimEnd()}\n`,
LOCAL_AUTHOR,
LOCAL_AUTHOR_EMAIL,
`Update semantic-layer source: ${input.sourceName}`,
);
return { success: true, sourceName: input.sourceName, yaml: `${yaml.trimEnd()}\n` };
},
async validate(input) {
if (options.semanticLayerCompute) {
const connectionId = assertSafeConnectionId(input.connectionId);
const result = await options.semanticLayerCompute.validateSources({
sources: await loadComputableSources(project, connectionId),
dialect: dialectForDriver(project.config.connections[connectionId]?.driver),
recentlyTouched: input.names,
});
return {
success: result.valid,
errors: result.errors,
warnings: result.warnings,
};
}
const names = new Set(input.names ?? []);
const paths = await listSlPaths(project, input.connectionId);
const errors: string[] = [];
for (const path of paths) {
const sourceName = sourceNameFromPath(path);
if (names.size > 0 && !names.has(sourceName)) {
continue;
}
try {
const raw = await project.fileStore.readFile(path);
errors.push(...validateSourceRecord(sourceName, parseYamlRecord(raw.content)));
} catch (error) {
errors.push(`${sourceName}: ${error instanceof Error ? error.message : String(error)}`);
}
}
return {
success: errors.length === 0,
errors,
warnings: [SL_SHAPE_WARNING],
};
},
async query(input) {
async query(input, executionOptions) {
if (!options.semanticLayerCompute) {
throw new Error(
'sl_query requires a semantic-layer query adapter. Local stdio MCP exposes file-backed SL CRUD only.',
);
throw new Error('sl_query requires a semantic-layer query adapter.');
}
return compileLocalSlQuery(project, {
connectionId: input.connectionId,
@ -629,6 +210,7 @@ export function createLocalProjectMcpContextPorts(
execute: Boolean(options.queryExecutor),
maxRows: input.query.limit,
queryExecutor: options.queryExecutor,
onProgress: executionOptions?.onProgress,
});
},
},
@ -651,114 +233,8 @@ export function createLocalProjectMcpContextPorts(
if (options.sqlAnalysis && options.localScan?.createConnector) {
ports.sqlExecution = {
async execute(input) {
return executeValidatedReadOnlySql(project, options, input);
},
};
}
if (options.localIngest) {
ports.ingest = {
async trigger(input) {
const sourceDir = localIngestSourceDir(input.config);
if (input.adapter === 'metabase' && !sourceDir) {
const result = await (options.localIngest?.runLocalMetabaseIngest ?? runLocalMetabaseIngest)({
project,
adapters: options.localIngest?.adapters ?? createDefaultLocalIngestAdapters(project),
metabaseConnectionId: input.connectionId,
trigger: input.trigger,
jobIdFactory: options.localIngest?.jobIdFactory,
pullConfigOptions: options.localIngest?.pullConfigOptions,
agentRunner: options.localIngest?.agentRunner,
llmProvider: options.localIngest?.llmProvider,
memoryModel: options.localIngest?.memoryModel,
semanticLayerCompute: options.localIngest?.semanticLayerCompute ?? options.semanticLayerCompute,
queryExecutor: options.localIngest?.queryExecutor ?? options.queryExecutor,
logger: options.localIngest?.logger,
});
return {
runId: `metabase-fanout:${result.metabaseConnectionId}`,
jobId: undefined,
reportId: undefined,
fanout: {
status: result.status,
children: result.children.map((child) => ({
runId: child.report.runId,
jobId: child.report.jobId,
reportId: child.report.id,
targetConnectionId: child.targetConnectionId,
metabaseDatabaseId: child.metabaseDatabaseId,
})),
},
};
}
const executeLocalIngest = options.localIngest?.runLocalIngest ?? runLocalIngest;
const result = await executeLocalIngest({
project,
adapters: options.localIngest?.adapters ?? createDefaultLocalIngestAdapters(project),
adapter: input.adapter,
connectionId: input.connectionId,
sourceDir,
pullConfigOptions: options.localIngest?.pullConfigOptions,
trigger: input.trigger,
jobId: options.localIngest?.jobIdFactory?.(),
agentRunner: options.localIngest?.agentRunner,
llmProvider: options.localIngest?.llmProvider,
memoryModel: options.localIngest?.memoryModel,
semanticLayerCompute: options.localIngest?.semanticLayerCompute ?? options.semanticLayerCompute,
queryExecutor: options.localIngest?.queryExecutor ?? options.queryExecutor,
logger: options.localIngest?.logger,
});
return {
runId: result.report.runId,
jobId: result.report.jobId,
reportId: result.report.id,
};
},
async status(input) {
const report = await getLocalIngestStatus(project, input.runId);
return report ? statusFromIngestReport(report) : null;
},
async report(input) {
return getLocalIngestStatus(project, input.runId);
},
async replay(input) {
const report = await getLocalIngestStatus(project, input.runId);
return report ? ingestReportToMemoryFlowReplay(report) : null;
},
};
}
if (options.localScan) {
ports.scan = {
async trigger(input) {
return runLocalScan({
project,
connectionId: input.connectionId,
mode: input.mode,
detectRelationships: input.detectRelationships,
dryRun: input.dryRun,
trigger: 'mcp',
adapters: options.localScan?.adapters,
databaseIntrospectionUrl: options.localScan?.databaseIntrospectionUrl,
createConnector: options.localScan?.createConnector,
jobId: options.localScan?.jobIdFactory?.(),
now: options.localScan?.now,
});
},
async status(input) {
return getLocalScanStatus(project, input.runId);
},
async report(input) {
return getLocalScanReport(project, input.runId);
},
async listArtifacts(input) {
const report = await getLocalScanReport(project, input.runId);
return report ? listArtifactsForReport(project, input.runId, report) : null;
},
async readArtifact(input) {
return readScanArtifact(project, input.runId, input.path);
async execute(input, executionOptions) {
return executeValidatedReadOnlySql(project, options, input, executionOptions?.onProgress);
},
};
}

1027
packages/context/src/mcp/server.test.ts
View file

File diff suppressed because it is too large Load diff

68
packages/context/src/mcp/server.ts
View file

@ -1,71 +1,8 @@
import { randomUUID } from 'node:crypto';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod';
import type { MemoryAgentInput } from '../memory/index.js';
import { jsonErrorToolResult, jsonToolResult, registerKtxContextTools } from './context-tools.js';
import type { KtxMcpServerDeps, KtxMcpServerLike, MemoryCapturePort } from './types.js';
const memoryCaptureInputSchema = {
userMessage: z.string().min(1).describe('The user message that may contain durable knowledge.'),
assistantMessage: z.string().optional().describe('The assistant response that concluded the exchange.'),
connectionId: z.string().min(1).optional().describe('Optional connection id for semantic-layer capture.'),
};
const memoryCaptureStatusInputSchema = {
runId: z.string().min(1).describe('The memory capture run id returned by memory_capture.'),
};
function registerMemoryCaptureTools(deps: {
server: KtxMcpServerLike;
memoryCapture: MemoryCapturePort;
userContext: KtxMcpServerDeps['userContext'];
}): void {
deps.server.registerTool(
'memory_capture',
{
title: 'Memory Capture',
description:
'Capture durable knowledge and semantic-layer updates from the final user/assistant exchange. Returns a run id for polling.',
inputSchema: memoryCaptureInputSchema,
},
async (input) => {
const captureInput: MemoryAgentInput = {
userId: deps.userContext.userId,
chatId: `mcp-${randomUUID()}`,
userMessage: String(input.userMessage),
assistantMessage: typeof input.assistantMessage === 'string' ? input.assistantMessage : undefined,
connectionId: typeof input.connectionId === 'string' ? input.connectionId : undefined,
sourceType: 'external_ingest',
};
const result = await deps.memoryCapture.capture(captureInput);
return jsonToolResult(result);
},
);
deps.server.registerTool(
'memory_capture_status',
{
title: 'Memory Capture Status',
description: 'Read the current or final status for a memory capture run.',
inputSchema: memoryCaptureStatusInputSchema,
},
async (input) => {
const runId = String(input.runId);
const status = await deps.memoryCapture.status(runId);
return status ? jsonToolResult(status) : jsonErrorToolResult(`Memory capture run "${runId}" was not found.`);
},
);
}
import { registerKtxContextTools } from './context-tools.js';
import type { KtxMcpServerDeps, KtxMcpServerLike } from './types.js';
export function createKtxMcpServer(deps: KtxMcpServerDeps): KtxMcpServerDeps['server'] {
if (deps.memoryCapture) {
registerMemoryCaptureTools({
server: deps.server,
memoryCapture: deps.memoryCapture,
userContext: deps.userContext,
});
}
if (deps.contextTools) {
registerKtxContextTools({
server: deps.server,
@ -86,7 +23,6 @@ export function createDefaultKtxMcpServer(
});
createKtxMcpServer({
server: server as KtxMcpServerLike,
memoryCapture: deps.memoryCapture,
userContext: deps.userContext,
contextTools: deps.contextTools,
});

275
packages/context/src/mcp/types.ts
View file

@ -1,16 +1,7 @@
import type { IngestReportSnapshot, MemoryFlowReplayInput, TableUsageOutput } from '../ingest/index.js';
import type { MemoryCaptureService } from '../memory/index.js';
import type { MemoryIngestService } from '../memory/index.js';
import type { KtxEntityDetailsInput, KtxEntityDetailsResponse } from '../scan/entity-details.js';
import type { KtxScanMode, KtxScanReport } from '../scan/index.js';
import type { KtxDiscoverDataInput, KtxDiscoverDataResponse } from '../search/index.js';
import type {
KtxDictionarySearchInput,
KtxDictionarySearchResponse,
SemanticLayerQueryInput,
SlDictionaryMatch,
SlSearchLaneSummary,
SlSearchMatchReason,
} from '../sl/index.js';
import type { KtxDictionarySearchInput, KtxDictionarySearchResponse, SemanticLayerQueryInput } from '../sl/index.js';
import type { WikiSearchLaneSummary, WikiSearchMatchReason } from '../wiki/index.js';
export interface KtxMcpTextContent {
@ -18,15 +9,38 @@ export interface KtxMcpTextContent {
text: string;
}
export interface KtxMcpToolResult<T extends object = object> {
export type NonArrayObject = object & { length?: never };
export interface KtxMcpToolResult<T extends NonArrayObject = NonArrayObject> {
content: KtxMcpTextContent[];
structuredContent?: T;
isError?: true;
}
export interface MemoryCapturePort {
capture: MemoryCaptureService['capture'];
status: MemoryCaptureService['status'];
interface KtxMcpProgressEvent {
progress: number;
total?: number;
message: string;
}
export type KtxMcpProgressCallback = (event: KtxMcpProgressEvent) => void | Promise<void>;
export interface KtxMcpToolHandlerContext {
_meta?: { progressToken?: string | number; [key: string]: unknown };
sendNotification?: (notification: {
method: 'notifications/progress';
params: {
progressToken: string | number;
progress: number;
total?: number;
message?: string;
};
}) => Promise<void>;
}
export interface MemoryIngestPort {
ingest: MemoryIngestService['ingest'];
status: MemoryIngestService['status'];
}
export interface KtxMcpUserContext {
@ -40,8 +54,10 @@ export interface KtxMcpServerLike {
title?: string;
description?: string;
inputSchema: unknown;
outputSchema?: unknown;
annotations?: Record<string, unknown>;
},
handler: (input: Record<string, unknown>) => Promise<unknown>,
handler: (input: Record<string, unknown>, context?: KtxMcpToolHandlerContext) => Promise<unknown>,
): void;
}
@ -51,18 +67,8 @@ export interface KtxConnectionSummary {
connectionType: string;
}
export interface KtxConnectionTestResponse {
id: string;
connectionType: string;
ok: boolean;
tableCount: number | null;
message: string;
warnings: string[];
}
export interface KtxConnectionsMcpPort {
list(): Promise<KtxConnectionSummary[]>;
test?(input: { connectionId: string }): Promise<KtxConnectionTestResponse | null>;
}
export interface KtxKnowledgeSearchResult {
@ -90,62 +96,9 @@ export interface KtxKnowledgePage {
slRefs?: string[];
}
interface KtxHistoricSqlKnowledgeUsage {
executions: number;
distinct_users: number;
first_seen: string;
last_seen: string;
p50_runtime_ms: number | null;
p95_runtime_ms: number | null;
error_rate: number;
rows_produced?: number;
}
export interface KtxKnowledgeWriteResponse {
success: boolean;
key: string;
action: 'created' | 'updated';
}
export interface KtxKnowledgeMcpPort {
search(input: { userId: string; query: string; limit: number }): Promise<KtxKnowledgeSearchResponse>;
read(input: { userId: string; key: string }): Promise<KtxKnowledgePage | null>;
write(input: {
userId: string;
key: string;
summary: string;
content: string;
tags?: string[];
refs?: string[];
slRefs?: string[];
source?: string;
intent?: string;
tables?: string[];
representativeSql?: string;
usage?: KtxHistoricSqlKnowledgeUsage;
fingerprints?: string[];
}): Promise<KtxKnowledgeWriteResponse>;
}
export interface KtxSemanticLayerSourceSummary {
connectionId: string;
connectionName: string;
name: string;
description?: string;
columnCount: number;
measureCount: number;
joinCount: number;
frequencyTier?: TableUsageOutput['frequencyTier'];
snippet?: string;
score?: number;
matchReasons?: SlSearchMatchReason[];
dictionaryMatches?: SlDictionaryMatch[];
lanes?: SlSearchLaneSummary[];
}
export interface KtxSemanticLayerListResponse {
sources: KtxSemanticLayerSourceSummary[];
totalSources: number;
}
export interface KtxSemanticLayerReadResponse {
@ -153,21 +106,6 @@ export interface KtxSemanticLayerReadResponse {
yaml: string;
}
export interface KtxSemanticLayerWriteResponse {
success: boolean;
sourceName: string;
yaml?: string;
errors?: string[];
warnings?: string[];
commitHash?: string;
}
export interface KtxSemanticLayerValidationResponse {
success: boolean;
errors: string[];
warnings: string[];
}
export interface KtxSemanticLayerQueryResponse {
sql: string;
headers: string[];
@ -177,143 +115,11 @@ export interface KtxSemanticLayerQueryResponse {
}
export interface KtxSemanticLayerMcpPort {
listSources(input: { connectionId?: string; query?: string }): Promise<KtxSemanticLayerListResponse>;
readSource(input: { connectionId: string; sourceName: string }): Promise<KtxSemanticLayerReadResponse | null>;
writeSource(input: {
connectionId: string;
sourceName: string;
yaml?: string;
source?: Record<string, unknown>;
delete?: boolean;
}): Promise<KtxSemanticLayerWriteResponse>;
validate(input: { connectionId: string; names?: string[] }): Promise<KtxSemanticLayerValidationResponse>;
query(input: { connectionId?: string; query: SemanticLayerQueryInput }): Promise<KtxSemanticLayerQueryResponse>;
}
export type KtxIngestTriggerKind = 'upload' | 'scheduled_pull' | 'manual_resync';
interface KtxIngestTriggerFanoutChild {
runId: string;
jobId: string;
reportId: string;
targetConnectionId: string;
metabaseDatabaseId: number;
}
export interface KtxIngestTriggerResponse {
runId: string;
jobId?: string;
reportId?: string;
fanout?: {
status: 'all_succeeded' | 'partial_failure' | 'all_failed';
children: KtxIngestTriggerFanoutChild[];
};
}
export interface KtxIngestDiffSummary {
added: number;
modified: number;
deleted: number;
unchanged: number;
}
export interface KtxIngestWorkUnitSummary {
unitKey: string;
rawFiles: string[];
peerFileIndex: string[];
dependencyPaths: string[];
}
export interface KtxIngestStatusResponse {
runId: string;
jobId?: string;
reportId?: string;
status: string;
stage?: string;
progress?: number;
errors?: string[];
done: boolean;
adapter?: string;
connectionId?: string;
sourceDir?: string | null;
syncId?: string;
startedAt?: string;
completedAt?: string;
previousRunId?: string | null;
diffSummary?: KtxIngestDiffSummary;
workUnitCount?: number;
rawFileCount?: number;
workUnits?: KtxIngestWorkUnitSummary[];
evictionDeletedRawPaths?: string[];
}
export interface KtxIngestMcpPort {
trigger(input: {
adapter: string;
connectionId: string;
config?: unknown;
trigger: KtxIngestTriggerKind;
}): Promise<KtxIngestTriggerResponse>;
status(input: { runId: string }): Promise<KtxIngestStatusResponse | null>;
report?(input: { runId: string }): Promise<IngestReportSnapshot | null>;
replay?(input: { runId: string }): Promise<MemoryFlowReplayInput | null>;
}
interface KtxScanTriggerResponse {
runId: string;
status: 'done';
done: true;
connectionId: string;
mode: KtxScanMode;
dryRun: boolean;
syncId: string;
report: KtxScanReport;
}
interface KtxScanStatusResponse {
runId: string;
status: string;
done: boolean;
connectionId: string;
mode: KtxScanMode;
dryRun: boolean;
syncId: string;
progress: number;
startedAt: string;
completedAt: string;
reportPath: string | null;
warnings: KtxScanReport['warnings'];
}
export type KtxScanArtifactType = 'report' | 'raw_source' | 'manifest_shard' | 'enrichment_artifact';
export interface KtxScanArtifactSummary {
path: string;
type: KtxScanArtifactType;
size?: number;
}
export interface KtxScanArtifactListResponse {
runId: string;
artifacts: KtxScanArtifactSummary[];
}
export interface KtxScanArtifactReadResponse extends KtxScanArtifactSummary {
runId: string;
content: string;
}
export interface KtxScanMcpPort {
trigger(input: {
connectionId: string;
mode?: KtxScanMode;
detectRelationships: boolean;
dryRun: boolean;
}): Promise<KtxScanTriggerResponse>;
status(input: { runId: string }): Promise<KtxScanStatusResponse | null>;
report(input: { runId: string }): Promise<KtxScanReport | null>;
listArtifacts?(input: { runId: string }): Promise<KtxScanArtifactListResponse | null>;
readArtifact?(input: { runId: string; path: string }): Promise<KtxScanArtifactReadResponse | null>;
query(
input: { connectionId?: string; query: SemanticLayerQueryInput },
options?: { onProgress?: KtxMcpProgressCallback },
): Promise<KtxSemanticLayerQueryResponse>;
}
export interface KtxEntityDetailsMcpPort {
@ -336,7 +142,10 @@ export interface KtxSqlExecutionResponse {
}
export interface KtxSqlExecutionMcpPort {
execute(input: { connectionId: string; sql: string; maxRows: number }): Promise<KtxSqlExecutionResponse>;
execute(
input: { connectionId: string; sql: string; maxRows: number },
options?: { onProgress?: KtxMcpProgressCallback },
): Promise<KtxSqlExecutionResponse>;
}
export interface KtxMcpContextPorts {
@ -347,13 +156,11 @@ export interface KtxMcpContextPorts {
dictionarySearch?: KtxDictionarySearchMcpPort;
discover?: KtxDiscoverDataMcpPort;
sqlExecution?: KtxSqlExecutionMcpPort;
ingest?: KtxIngestMcpPort;
scan?: KtxScanMcpPort;
memoryIngest?: MemoryIngestPort;
}
export interface KtxMcpServerDeps {
server: KtxMcpServerLike;
memoryCapture?: MemoryCapturePort;
userContext: KtxMcpUserContext;
contextTools?: KtxMcpContextPorts;
}

10
packages/context/src/memory/index.ts
View file

@ -8,13 +8,13 @@ export {
stepBudgetFor,
} from './capture-signals.js';
export { MemoryAgentService } from './memory-agent.service.js';
export { createLocalProjectMemoryCapture, type CreateLocalProjectMemoryCaptureOptions } from './local-memory.js';
export { createLocalProjectMemoryIngest, type CreateLocalProjectMemoryIngestOptions } from './local-memory.js';
export { LocalMemoryRunStore, type LocalMemoryRunStoreOptions } from './local-memory-runs.js';
export {
MemoryCaptureService,
type MemoryCaptureServiceDeps,
type MemoryCaptureStartResult,
type MemoryCaptureStatus,
MemoryIngestService,
type MemoryIngestServiceDeps,
type MemoryIngestStartResult,
type MemoryIngestStatus,
type MemoryRunRecord,
type MemoryRunStatus,
type MemoryRunStorePort,

20
packages/context/src/memory/local-memory.test.ts
View file

@ -3,7 +3,7 @@ import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { initKtxProject } from '../project/index.js';
import { createLocalProjectMemoryCapture } from './local-memory.js';
import { createLocalProjectMemoryIngest } from './local-memory.js';
import { LocalMemoryRunStore } from './local-memory-runs.js';
vi.mock('ai', () => ({
@ -77,7 +77,7 @@ describe('LocalMemoryRunStore', () => {
});
});
describe('createLocalProjectMemoryCapture', () => {
describe('createLocalProjectMemoryIngest', () => {
let tempDir: string;
beforeEach(async () => {
@ -110,13 +110,13 @@ describe('createLocalProjectMemoryCapture', () => {
},
};
const capture = createLocalProjectMemoryCapture(project, {
const ingest = createLocalProjectMemoryIngest(project, {
agentRunner: agentRunner as never,
runIdFactory: () => 'memory-run-1',
});
await expect(
capture.capture({
ingest.ingest({
userId: 'local-user',
chatId: 'chat-1',
userMessage: 'define revenue as paid order value net of refunds',
@ -124,12 +124,12 @@ describe('createLocalProjectMemoryCapture', () => {
sourceType: 'external_ingest',
}),
).resolves.toEqual({ runId: 'memory-run-1' });
await capture.waitForRun('memory-run-1');
await ingest.waitForRun('memory-run-1');
await expect(access(join(project.projectDir, '.ktx/db.sqlite'))).resolves.toBeUndefined();
await expectPathMissing(join(project.projectDir, '.ktx/memory-runs/memory-run-1.json'));
await expect(capture.status('memory-run-1')).resolves.toMatchObject({
await expect(ingest.status('memory-run-1')).resolves.toMatchObject({
runId: 'memory-run-1',
status: 'done',
done: true,
@ -172,12 +172,12 @@ describe('createLocalProjectMemoryCapture', () => {
},
};
const capture = createLocalProjectMemoryCapture(project, {
const ingest = createLocalProjectMemoryIngest(project, {
agentRunner: agentRunner as never,
runIdFactory: () => 'memory-run-2',
});
await capture.capture({
await ingest.ingest({
userId: 'local-user',
chatId: 'chat-2',
userMessage: 'going forward define orders count as count of public orders',
@ -185,12 +185,12 @@ describe('createLocalProjectMemoryCapture', () => {
connectionId: 'warehouse',
sourceType: 'external_ingest',
});
await capture.waitForRun('memory-run-2');
await ingest.waitForRun('memory-run-2');
await expect(access(join(project.projectDir, '.ktx/db.sqlite'))).resolves.toBeUndefined();
await expectPathMissing(join(project.projectDir, '.ktx/memory-runs/memory-run-2.json'));
await expect(capture.status('memory-run-2')).resolves.toMatchObject({
await expect(ingest.status('memory-run-2')).resolves.toMatchObject({
runId: 'memory-run-2',
status: 'done',
captured: { wiki: [], sl: ['orders'], xrefs: [] },

16
packages/context/src/memory/local-memory.ts
View file

@ -47,7 +47,7 @@ import {
} from '../wiki/index.js';
import { LocalMemoryRunStore } from './local-memory-runs.js';
import { MemoryAgentService } from './memory-agent.service.js';
import { MemoryCaptureService } from './memory-runs.js';
import { MemoryIngestService } from './memory-runs.js';
import type {
MemoryConnectionPort,
MemoryFileStorePort,
@ -60,9 +60,9 @@ import type {
const promptsDir = fileURLToPath(new URL('../../prompts', import.meta.url));
const skillsDir = fileURLToPath(new URL('../../skills', import.meta.url));
const LOCAL_AUTHOR = { name: 'KTX Local', email: 'local@ktx.local' };
const LOCAL_SHAPE_WARNING = 'Local memory capture validates semantic-layer YAML shape only.';
const LOCAL_SHAPE_WARNING = 'Local memory ingest validates semantic-layer YAML shape only.';
export interface CreateLocalProjectMemoryCaptureOptions {
export interface CreateLocalProjectMemoryIngestOptions {
llmProvider?: KtxLlmProvider;
agentRunner?: AgentRunnerService;
memoryModel?: string;
@ -72,10 +72,10 @@ export interface CreateLocalProjectMemoryCaptureOptions {
logger?: KtxLogger;
}
export function createLocalProjectMemoryCapture(
export function createLocalProjectMemoryIngest(
project: KtxLocalProject,
options: CreateLocalProjectMemoryCaptureOptions = {},
): MemoryCaptureService {
options: CreateLocalProjectMemoryIngestOptions = {},
): MemoryIngestService {
const logger = options.logger ?? noopLogger;
const rootFileStore = new LocalMemoryFileStore(project.fileStore);
const embedding = new NoopEmbeddingPort();
@ -137,7 +137,7 @@ export function createLocalProjectMemoryCapture(
toolsetFactory,
logger,
});
return new MemoryCaptureService({
return new MemoryIngestService({
memoryAgent,
runs: new LocalMemoryRunStore({ projectDir: project.projectDir, idFactory: options.runIdFactory }),
});
@ -145,7 +145,7 @@ export function createLocalProjectMemoryCapture(
function requireLlmProvider(provider: KtxLlmProvider | null | undefined): KtxLlmProvider {
if (!provider) {
throw new Error('createLocalProjectMemoryCapture requires llm.provider.backend or an injected agentRunner');
throw new Error('createLocalProjectMemoryIngest requires llm.provider.backend or an injected agentRunner');
}
return provider;
}

54
packages/context/src/memory/memory-runs.test.ts
View file

@ -1,6 +1,6 @@
import { describe, expect, it, vi } from 'vitest';
import type { MemoryAgentInput, MemoryAgentResult, MemoryAgentService } from './index.js';
import { MemoryCaptureService, type MemoryRunStorePort } from './memory-runs.js';
import { MemoryIngestService, type MemoryRunStorePort } from './memory-runs.js';
class InMemoryRunStore implements MemoryRunStorePort {
readonly rows = new Map<
@ -74,32 +74,32 @@ function deferred<T>() {
}
function buildService(): {
capture: MemoryCaptureService;
ingest: MemoryIngestService;
store: InMemoryRunStore;
ingest: ReturnType<typeof vi.fn>;
memoryAgentIngest: ReturnType<typeof vi.fn>;
run: ReturnType<typeof deferred<MemoryAgentResult>>;
} {
const store = new InMemoryRunStore();
const run = deferred<MemoryAgentResult>();
const ingest = vi.fn<MemoryAgentService['ingest']>().mockReturnValue(run.promise);
const memoryAgent = { ingest };
const memoryAgentIngest = vi.fn<MemoryAgentService['ingest']>().mockReturnValue(run.promise);
const memoryAgent = { ingest: memoryAgentIngest };
return {
capture: new MemoryCaptureService({ memoryAgent, runs: store }),
ingest: new MemoryIngestService({ memoryAgent, runs: store }),
store,
ingest,
memoryAgentIngest,
run,
};
}
describe('MemoryCaptureService', () => {
it('creates a run, executes memory capture, and stores a done summary', async () => {
describe('MemoryIngestService', () => {
it('creates a run, executes memory ingest, and stores a done summary', async () => {
const result: MemoryAgentResult = {
signalDetected: true,
actions: [{ target: 'wiki', type: 'created', key: 'revenue', detail: 'captured revenue definition' }],
skillsLoaded: ['wiki_capture'],
commitHash: 'abc123',
};
const { capture, store, ingest, run } = buildService();
const { ingest, store, memoryAgentIngest, run } = buildService();
const input: MemoryAgentInput = {
userId: 'user-1',
@ -109,21 +109,21 @@ describe('MemoryCaptureService', () => {
connectionId: '00000000-0000-0000-0000-000000000001',
};
const started = await capture.capture(input);
const started = await ingest.ingest(input);
expect(started.runId).toBe('run-1');
expect(ingest).toHaveBeenCalledWith(input);
await expect(capture.status(started.runId)).resolves.toMatchObject({
expect(memoryAgentIngest).toHaveBeenCalledWith(input);
await expect(ingest.status(started.runId)).resolves.toMatchObject({
runId: 'run-1',
status: 'running',
stage: 'capturing',
stage: 'ingesting',
done: false,
});
run.resolve(result);
await capture.waitForRun(started.runId);
await ingest.waitForRun(started.runId);
const status = await capture.status(started.runId);
const status = await ingest.status(started.runId);
expect(status).toEqual({
runId: 'run-1',
stage: 'done',
@ -142,10 +142,10 @@ describe('MemoryCaptureService', () => {
expect(store.rows.get('run-1')?.inputHash).toHaveLength(64);
});
it('stores no-signal captures as done with empty captured arrays', async () => {
const { capture, run } = buildService();
it('stores no-signal ingests as done with empty captured arrays', async () => {
const { ingest, run } = buildService();
const started = await capture.capture({
const started = await ingest.ingest({
userId: 'user-1',
chatId: 'chat-2',
userMessage: 'Thanks.',
@ -157,9 +157,9 @@ describe('MemoryCaptureService', () => {
skillsLoaded: [],
commitHash: null,
});
await capture.waitForRun(started.runId);
await ingest.waitForRun(started.runId);
await expect(capture.status(started.runId)).resolves.toMatchObject({
await expect(ingest.status(started.runId)).resolves.toMatchObject({
done: true,
status: 'done',
captured: { wiki: [], sl: [], xrefs: [] },
@ -172,16 +172,16 @@ describe('MemoryCaptureService', () => {
const memoryAgent = {
ingest: vi.fn<MemoryAgentService['ingest']>().mockRejectedValue(new Error('LLM provider missing')),
};
const capture = new MemoryCaptureService({ memoryAgent, runs: store });
const ingest = new MemoryIngestService({ memoryAgent, runs: store });
const started = await capture.capture({
const started = await ingest.ingest({
userId: 'user-1',
chatId: 'chat-3',
userMessage: 'Remember this.',
});
await capture.waitForRun(started.runId);
await ingest.waitForRun(started.runId);
await expect(capture.status(started.runId)).resolves.toMatchObject({
await expect(ingest.status(started.runId)).resolves.toMatchObject({
done: true,
status: 'error',
stage: 'error',
@ -191,8 +191,8 @@ describe('MemoryCaptureService', () => {
});
it('returns null for an unknown run id', async () => {
const { capture } = buildService();
const { ingest } = buildService();
await expect(capture.status('missing')).resolves.toBeNull();
await expect(ingest.status('missing')).resolves.toBeNull();
});
});

22
packages/context/src/memory/memory-runs.ts
View file

@ -21,16 +21,16 @@ export interface MemoryRunStorePort {
findById(id: string): Promise<MemoryRunRecord | null>;
}
export interface MemoryCaptureServiceDeps {
export interface MemoryIngestServiceDeps {
memoryAgent: Pick<MemoryAgentService, 'ingest'>;
runs: MemoryRunStorePort;
}
export interface MemoryCaptureStartResult {
export interface MemoryIngestStartResult {
runId: string;
}
export interface MemoryCaptureStatus {
export interface MemoryIngestStatus {
runId: string;
status: MemoryRunStatus;
stage: string;
@ -55,7 +55,7 @@ function inputHash(input: MemoryAgentInput): string {
return createHash('sha256').update(stableInput).digest('hex');
}
function capturedKeys(actions: MemoryAction[]): MemoryCaptureStatus['captured'] {
function capturedKeys(actions: MemoryAction[]): MemoryIngestStatus['captured'] {
const wiki = new Set<string>();
const sl = new Set<string>();
const xrefs = new Set<string>();
@ -78,20 +78,20 @@ function capturedKeys(actions: MemoryAction[]): MemoryCaptureStatus['captured']
};
}
export class MemoryCaptureService {
export class MemoryIngestService {
private readonly inFlight = new Map<string, Promise<void>>();
constructor(private readonly deps: MemoryCaptureServiceDeps) {}
constructor(private readonly deps: MemoryIngestServiceDeps) {}
async capture(input: MemoryAgentInput): Promise<MemoryCaptureStartResult> {
async ingest(input: MemoryAgentInput): Promise<MemoryIngestStartResult> {
const row = await this.deps.runs.createRunning({
inputHash: inputHash(input),
chatId: input.chatId,
});
await this.deps.runs.markRunning(row.id, 'capturing');
await this.deps.runs.markRunning(row.id, 'ingesting');
const run = this.runCapture(row.id, input);
const run = this.runIngest(row.id, input);
this.inFlight.set(row.id, run);
run.finally(() => this.inFlight.delete(row.id)).catch(() => undefined);
@ -102,7 +102,7 @@ export class MemoryCaptureService {
await this.inFlight.get(runId);
}
private async runCapture(runId: string, input: MemoryAgentInput): Promise<void> {
private async runIngest(runId: string, input: MemoryAgentInput): Promise<void> {
try {
const outputSummary = await this.deps.memoryAgent.ingest(input);
await this.deps.runs.markDone(runId, outputSummary);
@ -111,7 +111,7 @@ export class MemoryCaptureService {
}
}
async status(runId: string): Promise<MemoryCaptureStatus | null> {
async status(runId: string): Promise<MemoryIngestStatus | null> {
const row = await this.deps.runs.findById(runId);
if (!row) {
return null;

77
packages/context/src/sl/local-query.test.ts
View file

@ -182,6 +182,46 @@ grain: []
});
});
it('strips authoring-only fields (usage, inherits_columns_from) before sending sources to the daemon', async () => {
await project.fileStore.writeFile(
'semantic-layer/warehouse/_schema/public.yaml',
`tables:
invoices:
table: public.invoices
columns:
- name: invoice_id
type: number
pk: true
- name: amount
type: number
usage:
narrative: Activation policy windows table for invoice analytics.
frequencyTier: mid
commonFilters:
- amount
commonGroupBys: []
commonJoins: []
staleSince: null
`,
'ktx',
'ktx@example.com',
'Add manifest shard with usage',
);
await compileLocalSlQuery(project, {
connectionId: 'warehouse',
query: { measures: ['sum(invoices.amount)'], dimensions: [] },
compute,
});
const lastCall = (compute.query as ReturnType<typeof vi.fn>).mock.calls.at(-1)?.[0];
const invoices = lastCall?.sources.find((s: Record<string, unknown>) => s.name === 'invoices');
expect(invoices).toBeDefined();
expect(invoices).not.toHaveProperty('usage');
expect(invoices).not.toHaveProperty('inherits_columns_from');
expect(invoices).not.toHaveProperty('source_type');
});
it('resolves the only configured connection when connectionId is omitted', async () => {
await compileLocalSlQuery(project, {
query: { measures: ['orders.order_count'], dimensions: [] },
@ -236,6 +276,43 @@ grain: []
});
});
it('emits progress while compiling and executing a local semantic-layer query', async () => {
const progress: Array<{ progress: number; message: string }> = [];
const queryExecutor = {
execute: vi.fn(async () => ({
headers: ['status', 'order_count'],
rows: [['paid', 2]],
totalRows: 1,
command: 'SELECT',
rowCount: 1,
})),
};
const result = await compileLocalSlQuery(project, {
connectionId: 'warehouse',
query: {
measures: ['orders.order_count'],
dimensions: ['orders.status'],
limit: 25,
},
compute,
execute: true,
maxRows: 10,
queryExecutor,
onProgress: (event) => {
progress.push({ progress: event.progress, message: event.message });
},
});
expect(result.totalRows).toBe(1);
expect(progress).toEqual([
{ progress: 0, message: 'Compiling query' },
{ progress: 0.3, message: 'Generating SQL' },
{ progress: 0.6, message: 'Executing' },
{ progress: 1, message: 'Fetched 1 rows' },
]);
});
it('requires a query executor for executed mode', async () => {
await expect(
compileLocalSlQuery(project, {

18
packages/context/src/sl/local-query.ts
View file

@ -1,7 +1,9 @@
import type { KtxSqlQueryExecutorPort } from '../connections/index.js';
import type { KtxSemanticLayerComputePort } from '../daemon/index.js';
import type { KtxMcpProgressCallback } from '../mcp/types.js';
import type { KtxLocalProject } from '../project/index.js';
import { loadLocalSlSourceRecords } from './local-sl.js';
import { toResolvedWire } from './semantic-layer.service.js';
import type { SemanticLayerQueryExecutionResult, SemanticLayerQueryInput } from './types.js';
const COMPILE_ONLY_REASON =
@ -14,6 +16,7 @@ export interface CompileLocalSlQueryOptions {
execute?: boolean;
maxRows?: number;
queryExecutor?: KtxSqlQueryExecutorPort;
onProgress?: KtxMcpProgressCallback;
}
export interface CompileLocalSlQueryResult extends SemanticLayerQueryExecutionResult {
@ -75,10 +78,10 @@ function resolveLocalConnectionId(project: KtxLocalProject, requested: string |
async function loadComputableSources(
project: KtxLocalProject,
connectionId: string,
): Promise<Record<string, unknown>[]> {
): Promise<ReturnType<typeof toResolvedWire>[]> {
return (await loadLocalSlSourceRecords(project, { connectionId: assertSafeConnectionId(connectionId) }))
.map((record) => ({ ...record.source }))
.filter((source) => source.table || source.sql);
.filter((record) => record.source.table || record.source.sql)
.map((record) => toResolvedWire(record.source));
}
function headersFromColumns(columns: Array<Record<string, unknown>>): string[] {
@ -91,15 +94,20 @@ export async function compileLocalSlQuery(
project: KtxLocalProject,
options: CompileLocalSlQueryOptions,
): Promise<CompileLocalSlQueryResult> {
await options.onProgress?.({ progress: 0, message: 'Compiling query' });
const connectionId = resolveLocalConnectionId(project, options.connectionId);
const dialect = dialectForDriver(project.config.connections[connectionId]?.driver);
const sources = await loadComputableSources(project, connectionId);
await options.onProgress?.({ progress: 0.3, message: 'Generating SQL' });
const response = await options.compute.query({
sources: await loadComputableSources(project, connectionId),
sources,
dialect,
query: options.query,
});
if (!options.execute) {
await options.onProgress?.({ progress: 1, message: 'Fetched 0 rows' });
return {
connectionId,
dialect: response.dialect,
@ -122,6 +130,7 @@ export async function compileLocalSlQuery(
}
const maxRows = options.maxRows ?? options.query.limit;
await options.onProgress?.({ progress: 0.6, message: 'Executing' });
const execution = await options.queryExecutor.execute({
connectionId,
projectDir: project.projectDir,
@ -129,6 +138,7 @@ export async function compileLocalSlQuery(
sql: response.sql,
maxRows,
});
await options.onProgress?.({ progress: 1, message: `Fetched ${execution.totalRows} rows` });
return {
connectionId,

8
pnpm-lock.yaml generated
View file

@ -112,6 +112,9 @@ importers:
commander:
specifier: 14.0.3
version: 14.0.3
fflate:
specifier: ^0.8.2
version: 0.8.2
ink:
specifier: ^7.0.2
version: 7.0.2(@types/react@19.2.14)(react@19.2.6)
@ -3191,6 +3194,9 @@ packages:
resolution: {integrity: sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==}
engines: {node: ^12.20 || >= 14.13}
fflate@0.8.2:
resolution: {integrity: sha512-cPJU47OaAoCbg0pBvzsgpTPhmhqI5eJjh/JIu8tPj5q+T7iLvW/JAYUqmE7KOB4R1ZyEhzBaIQpQpardBF5z8A==}
file-uri-to-path@1.0.0:
resolution: {integrity: sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==}
@ -8355,6 +8361,8 @@ snapshots:
node-domexception: 1.0.0
web-streams-polyfill: 3.3.3
fflate@0.8.2: {}
file-uri-to-path@1.0.0: {}
finalhandler@2.1.1: