Merge origin/main into copy-claude-code-backend-spec

This commit is contained in:
Andrey Avtomonov 2026-05-16 11:46:18 +02:00
commit 7287e4907b
59 changed files with 8221 additions and 3159 deletions

View file

@ -6,6 +6,8 @@
The context layer for analytics agents The context layer for analytics agents
</h1> </h1>
<p align="center">by Kaelio</p>
<p align="center"> <p align="center">
<a href="https://www.npmjs.com/package/@kaelio/ktx"><img src="https://img.shields.io/npm/v/@kaelio/ktx?style=flat-square&color=f97316" alt="npm version" /></a> <a href="https://www.npmjs.com/package/@kaelio/ktx"><img src="https://img.shields.io/npm/v/@kaelio/ktx?style=flat-square&color=f97316" alt="npm version" /></a>
<a href="https://codecov.io/gh/Kaelio/ktx"><img src="https://codecov.io/gh/Kaelio/ktx/branch/main/graph/badge.svg" alt="Codecov" /></a> <a href="https://codecov.io/gh/Kaelio/ktx"><img src="https://codecov.io/gh/Kaelio/ktx/branch/main/graph/badge.svg" alt="Codecov" /></a>
@ -40,6 +42,19 @@ ktx status
`ktx setup` creates or resumes a local KTX project, configures providers and `ktx setup` creates or resumes a local KTX project, configures providers and
connections, builds context, and installs agent integration. connections, builds context, and installs agent integration.
Example `ktx status` output after setup:
```text
KTX project: /home/user/analytics
Project ready: yes
LLM ready: yes (claude-sonnet-4-6)
Embeddings ready: yes (text-embedding-3-small)
Databases configured: yes (postgres-warehouse)
Context sources configured: yes (dbt-main)
KTX context built: yes
Agent integration ready: yes (codex:project)
```
## Common Commands ## Common Commands
| Command | Purpose | | Command | Purpose |
@ -92,6 +107,49 @@ ktx wiki search "refund policy" --json
ktx sl query --connection-id warehouse --measure orders.revenue --format sql ktx sl query --connection-id warehouse --measure orders.revenue --format sql
``` ```
During agent setup, choose **MCP tools + analytics skill** for client agents.
Choose **MCP tools + analytics skill + admin CLI skill** only when a developer
or operator agent also needs pinned `ktx` admin commands.
The analytics skill teaches client agents the MCP workflow: discover data,
prefer semantic-layer measures, inspect entity details before raw SQL, and
capture durable learnings. Admin CLI skills call `ktx` commands directly
through a skill file installed in your agent's config:
```bash
ktx sl query --measure orders.revenue --dimension orders.status --format sql
ktx wiki search "revenue definition"
ktx sl validate orders
```
Supported client agents: Claude Code, Claude Desktop, Codex, Cursor, OpenCode,
and clients that can use the printed MCP endpoint or `.agents` admin skills.
Claude Desktop setup registers a local `ktx mcp stdio` server in Claude
Desktop's config and generates `.ktx/agents/claude/ktx-plugin.zip` with the
analytics skill.
The release artifact manifest contains the public npm tarball and the bundled
`kaelio-ktx` runtime wheel. The `python/ktx-sl` and `python/ktx-daemon`
directories remain source packages for development, not public release
artifacts.
## Workspace packages
| Package | Purpose |
|---------|---------|
| `packages/cli` | CLI entry point |
| `packages/context` | Core context engine |
| `packages/llm` | LLM and embedding providers |
| `packages/connector-bigquery` | BigQuery scan connector |
| `packages/connector-clickhouse` | ClickHouse scan connector |
| `packages/connector-mysql` | MySQL scan connector |
| `packages/connector-postgres` | Postgres scan connector |
| `packages/connector-snowflake` | Snowflake scan connector |
| `packages/connector-sqlite` | SQLite scan connector |
| `packages/connector-sqlserver` | SQL Server scan connector |
| `python/ktx-sl` | Semantic-layer query planning |
| `python/ktx-daemon` | Portable compute service |
## Development ## Development
```bash ```bash

View file

@ -1,18 +1,26 @@
--- ---
title: Agent Clients title: Agent Clients
description: Set up KTX with Claude Code, Cursor, Codex, and OpenCode. description: Set up KTX with Claude Code, Claude Desktop, Cursor, Codex, and OpenCode.
--- ---
KTX integrates with coding agents through CLI skills and command files. These KTX exposes context to end-user agents through MCP tools. The CLI remains the
files teach agents to call public `ktx` commands directly from the terminal for admin surface for setup, ingest, status, daemon lifecycle, and debugging.
semantic-layer context and wiki knowledge.
Run `ktx setup` and select your agent targets, or configure manually using the Run `ktx setup` and select your client agent targets, or configure manually
snippets below. Setup pins generated skill files to the KTX CLI path that using the snippets below. Choose **MCP tools + analytics skill** for client
created them, so agents do not need `ktx` on `PATH`. agents. Choose **MCP tools + analytics skill + admin CLI skill** only when a
developer or operator agent also needs pinned `ktx` admin commands.
## Install with setup ## Install with setup
Start the MCP server before connecting an end-user agent:
```bash
ktx mcp start
```
Then install client integration:
```bash ```bash
ktx setup --agents ktx setup --agents
``` ```
@ -23,7 +31,8 @@ Use `--target` for one target:
ktx setup --agents --target codex ktx setup --agents --target codex
``` ```
Use `--global` only with `claude-code` or `codex`: Use `--global` only with `claude-code` or `codex`. Claude Desktop always
generates a project-local plugin ZIP:
```bash ```bash
ktx setup --agents --target claude-code --global ktx setup --agents --target claude-code --global
@ -34,19 +43,54 @@ KTX records installed files in `.ktx/agents/install-manifest.json`. That
manifest lets status checks report agent readiness and lets future cleanup manifest lets status checks report agent readiness and lets future cleanup
remove only files KTX installed. remove only files KTX installed.
The interactive command asks two questions:
```txt
◆ How should client agents connect to this KTX project?
│ ○ MCP tools + analytics skill
│ ○ MCP tools + analytics skill + admin CLI skill
◆ Which agent targets should KTX install?
│ ◻ Claude Code
│ ◻ Claude Desktop
│ ◻ Codex
│ ◻ Cursor
│ ◻ OpenCode
│ ◻ Universal .agents
```
When every selected target supports both project and global setup, the command
also asks where to install supported agent config:
```txt
◆ Where should KTX install supported agent config?
│ ○ Project
│ ○ Global
```
## Generated files ## Generated files
| Target | Project-scoped files | Global files | KTX writes MCP client configuration and an analytics skill by default. It writes
|--------|----------------------|--------------| admin CLI skills only when you choose **MCP tools + analytics skill + admin CLI
| Claude Code | `.claude/skills/ktx/SKILL.md`, `.claude/rules/ktx.md` | `~/.claude/skills/ktx/SKILL.md`, `~/.claude/rules/ktx.md` | skill**.
| Codex | `.agents/skills/ktx/SKILL.md`, `.codex/instructions/ktx.md` | `$CODEX_HOME/skills/ktx/SKILL.md`, `$CODEX_HOME/instructions/ktx.md` |
| Cursor | `.cursor/rules/ktx.mdc` | Not supported |
| OpenCode | `.opencode/commands/ktx.md` | Not supported |
| Universal `.agents` | `.agents/skills/ktx/SKILL.md` | Not supported |
Skill files list pinned `ktx` commands. Rule files tell the agent when KTX is | Target | MCP tools + analytics skill | Adds with admin CLI skill |
appropriate, such as data schemas, metrics, dimensions, database structure, and |--------|------------------------------|---------------------------|
SQL questions. | Claude Code | `.mcp.json`, `.claude/skills/ktx-analytics/SKILL.md` | `.claude/skills/ktx/SKILL.md`, `.claude/rules/ktx.md` |
| Claude Desktop | `~/Library/Application Support/Claude/claude_desktop_config.json` stdio entry + `.ktx/agents/claude/ktx-plugin.zip` with analytics skill | Adds `skills/ktx/SKILL.md` inside the plugin ZIP |
| Codex | Printed snippet for `~/.codex/config.toml`, `.agents/skills/ktx-analytics/SKILL.md` | `.agents/skills/ktx/SKILL.md`, `.codex/instructions/ktx.md` |
| Cursor | `.cursor/mcp.json`, `.cursor/rules/ktx-analytics.mdc` | `.cursor/rules/ktx.mdc` |
| OpenCode | Printed snippet for `opencode.json`, `.opencode/commands/ktx-analytics.md` | `.opencode/commands/ktx.md` |
| Universal `.agents` | Printed MCP endpoint, `.agents/skills/ktx-analytics/SKILL.md` | `.agents/skills/ktx/SKILL.md` |
MCP config gives agents access to KTX context tools such as discovery,
semantic-layer queries, wiki search, SQL execution, and memory ingest. The
analytics skill explains how to use those tools for semantic-layer-first
analysis. Optional admin skill and rule files list pinned CLI commands for
developer or operator agents.
## Claude Code ## Claude Code
@ -56,13 +100,16 @@ During setup, select **Claude Code** from the agent targets. KTX writes:
| Scope | Files | | Scope | Files |
|-------|-------| |-------|-------|
| Project | `.claude/skills/ktx/SKILL.md`, `.claude/rules/ktx.md` | | Project | `.mcp.json`, `.claude/skills/ktx-analytics/SKILL.md`; optional `.claude/skills/ktx/SKILL.md`, `.claude/rules/ktx.md` |
| Global | `~/.claude/skills/ktx/SKILL.md`, `~/.claude/rules/ktx.md` | | Global | `~/.claude.json`, `~/.claude/skills/ktx-analytics/SKILL.md`; optional `~/.claude/skills/ktx/SKILL.md`, `~/.claude/rules/ktx.md` |
Both project-scoped and global installations are supported. Both project-scoped and global installations are supported.
### Manual CLI skills configuration ### Manual CLI skills configuration
Use manual CLI skills only for developer or operator agents that need admin
commands. End-user data agents use MCP.
Create `.claude/skills/ktx/SKILL.md`: Create `.claude/skills/ktx/SKILL.md`:
```markdown title=".claude/skills/ktx/SKILL.md" ```markdown title=".claude/skills/ktx/SKILL.md"
@ -82,6 +129,7 @@ Available commands:
### Workflow tips ### Workflow tips
- Claude Code discovers skills automatically from `.claude/skills/`. - Claude Code discovers skills automatically from `.claude/skills/`.
- Claude Code reads MCP config from `.mcp.json` for project-scoped MCP tools.
- Claude rules in `.claude/rules/` tell Claude when KTX should be used. - Claude rules in `.claude/rules/` tell Claude when KTX should be used.
- Global installation makes KTX available in all projects without per-project setup. - Global installation makes KTX available in all projects without per-project setup.
- Keep generated skills committed only when your team wants project-local agent instructions in git. - Keep generated skills committed only when your team wants project-local agent instructions in git.
@ -96,13 +144,19 @@ During setup, select **Cursor** from the agent targets. KTX writes:
| Mode | File | | Mode | File |
|------|------| |------|------|
| CLI rules | `.cursor/rules/ktx.mdc` | | MCP tools + analytics skill | `.cursor/mcp.json`, `.cursor/rules/ktx-analytics.mdc` |
| Admin CLI rules | `.cursor/rules/ktx.mdc` |
Cursor supports project-scoped installation only. Cursor supports project-scoped installation only.
### Manual CLI rules configuration ### Manual CLI rules configuration
Create `.cursor/rules/ktx.mdc` with the same content structure as the Claude Code `SKILL.md` file. Cursor rules use the `.mdc` extension but support the same markdown command definitions. Use manual CLI rules only for developer or operator agents that need admin
commands. End-user data agents use MCP.
Create `.cursor/rules/ktx.mdc` with the same content structure as the Claude
Code `SKILL.md` file. Cursor rules use the `.mdc` extension but support the
same markdown command definitions.
### Workflow tips ### Workflow tips
@ -111,6 +165,37 @@ Create `.cursor/rules/ktx.mdc` with the same content structure as the Claude Cod
--- ---
## Claude Desktop
During setup, select **Claude Desktop** from the agent targets. KTX writes the
MCP server entry directly into Claude Desktop's config and generates a separate
plugin ZIP for the analytics skill:
- `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or
`%AppData%/Claude/claude_desktop_config.json` (Windows) gets an
`mcpServers.ktx` entry that runs the KTX MCP server over stdio via a local
launcher shim at `.ktx/agents/claude/ktx-plugin-runner.sh`. The shim locates
a usable Node.js (Volta, NVM, Homebrew, system) so Claude Desktop can spawn
the server without needing `node` in PATH.
- `.ktx/agents/claude/ktx-plugin.zip` contains the `ktx-analytics` skill (and
the admin `ktx` skill if you choose **MCP tools + analytics skill + admin
CLI skill**). Install the ZIP from Claude Desktop's plugin UI to load the
skill.
After `ktx setup`, restart Claude Desktop so it picks up the new MCP server
entry, then install the plugin ZIP. No daemon needs to be running — Claude
Desktop spawns the MCP server itself per session.
Claude Desktop does not introspect local stdio MCP servers, so the per-tool
"Connector"-style UI is not rendered for KTX. The tools are still callable
from any Claude Desktop chat.
If you move the KTX checkout or project directory, rerun `ktx setup --agents`
to refresh the absolute paths in `claude_desktop_config.json` and the launcher
shim, then reinstall the regenerated plugin ZIP.
---
## Codex ## Codex
### Install via `ktx setup` ### Install via `ktx setup`
@ -119,15 +204,19 @@ During setup, select **Codex** from the agent targets. KTX writes:
| Scope | Files | | Scope | Files |
|-------|-------| |-------|-------|
| Project | `.agents/skills/ktx/SKILL.md`, `.codex/instructions/ktx.md` | | Project | MCP snippet, `.agents/skills/ktx-analytics/SKILL.md`; optional `.agents/skills/ktx/SKILL.md`, `.codex/instructions/ktx.md` |
| Global | `$CODEX_HOME/skills/ktx/SKILL.md`, `$CODEX_HOME/instructions/ktx.md` | | Global | MCP snippet, `$CODEX_HOME/skills/ktx-analytics/SKILL.md`; optional `$CODEX_HOME/skills/ktx/SKILL.md`, `$CODEX_HOME/instructions/ktx.md` |
Both project-scoped and global installations are supported. `CODEX_HOME` Both project-scoped and global installations are supported. `CODEX_HOME`
defaults to `~/.codex`. defaults to `~/.codex`.
### Manual CLI skills configuration ### Manual CLI skills configuration
Create `.agents/skills/ktx/SKILL.md` with the same content structure as Claude Code's `SKILL.md`. Use manual CLI skills only for developer or operator agents that need admin
commands. End-user data agents use MCP.
Create `.agents/skills/ktx/SKILL.md` with the same content structure as Claude
Code's `SKILL.md`.
### Workflow tips ### Workflow tips
@ -146,13 +235,18 @@ During setup, select **OpenCode** from the agent targets. KTX writes:
| Mode | File | | Mode | File |
|------|------| |------|------|
| CLI commands | `.opencode/commands/ktx.md` | | MCP tools + analytics skill | Snippet for `opencode.json`, `.opencode/commands/ktx-analytics.md` |
| Admin CLI commands | `.opencode/commands/ktx.md` |
OpenCode supports project-scoped installation only. OpenCode supports project-scoped installation only.
### Manual CLI commands configuration ### Manual CLI commands configuration
Create `.opencode/commands/ktx.md` with the same command definitions as Claude Code's `SKILL.md`. Use manual CLI commands only for developer or operator agents that need admin
commands. End-user data agents use MCP.
Create `.opencode/commands/ktx.md` with the same command definitions as Claude
Code's `SKILL.md`.
### Workflow tips ### Workflow tips
@ -163,7 +257,7 @@ Create `.opencode/commands/ktx.md` with the same command definitions as Claude C
## Command reference ## Command reference
All supported agent clients call the same KTX CLI commands: Admin CLI skills call the same KTX CLI commands:
| Command | Description | | Command | Description |
|---------|-------------| |---------|-------------|
@ -183,9 +277,11 @@ All supported agent clients call the same KTX CLI commands:
## Comparison ## Comparison
| | Claude Code | Cursor | Codex | OpenCode | | | Claude Code | Claude Desktop | Cursor | Codex | OpenCode |
|---|---|---|---|---| |---|---|---|---|---|---|
| CLI skills | Yes | Yes (.mdc) | Yes | Yes | | MCP tools | Yes | Local stdio via `claude_desktop_config.json` | Yes | Snippet | Snippet |
| Global install | Yes | No | Yes | No | | Analytics skill | `.claude/skills/ktx-analytics/SKILL.md` | Included in plugin ZIP | `.cursor/rules/ktx-analytics.mdc` | `.agents/skills/ktx-analytics/SKILL.md` | `.opencode/commands/ktx-analytics.md` |
| Rule or instruction file | `.claude/rules/ktx.md` | `.cursor/rules/ktx.mdc` | `.codex/instructions/ktx.md` | `.opencode/commands/ktx.md` | | Admin CLI skills | Optional | Optional in plugin ZIP | Optional (.mdc) | Optional | Optional |
| Skill file | `.claude/skills/ktx/SKILL.md` | Not separate | `.agents/skills/ktx/SKILL.md` | Not separate | | Global install | Yes | Project-local ZIP | No | Yes | No |
| Rule or instruction file | `.claude/rules/ktx.md` | Plugin `SETUP.md` | `.cursor/rules/ktx.mdc` | `.codex/instructions/ktx.md` | `.opencode/commands/ktx.md` |
| Skill file | `.claude/skills/ktx/SKILL.md` | `skills/ktx/SKILL.md` in plugin ZIP | Not separate | `.agents/skills/ktx/SKILL.md` | Not separate |

View file

@ -1,6 +1,6 @@
/// <reference types="next" /> /// <reference types="next" />
/// <reference types="next/image-types/global" /> /// <reference types="next/image-types/global" />
import "./.next/dev/types/routes.d.ts"; import "./.next/types/routes.d.ts";
// NOTE: This file should not be edited // NOTE: This file should not be edited
// see https://nextjs.org/docs/app/api-reference/config/typescript for more information. // see https://nextjs.org/docs/app/api-reference/config/typescript for more information.

View file

@ -903,14 +903,16 @@ Add this test after the `dictionary_search` registration test:
limit: 5, limit: 5,
}), }),
).resolves.toMatchObject({ ).resolves.toMatchObject({
structuredContent: [ structuredContent: {
{ refs: [
kind: 'table', {
id: 'public.orders', kind: 'table',
connectionId: 'warehouse', id: 'public.orders',
tableRef: { catalog: null, db: 'public', name: 'orders' }, connectionId: 'warehouse',
}, tableRef: { catalog: null, db: 'public', name: 'orders' },
], },
],
},
}); });
expect(discover.search).toHaveBeenCalledWith({ expect(discover.search).toHaveBeenCalledWith({
query: 'orders', query: 'orders',

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -143,7 +143,9 @@ or `entity_details` tools.
} }
``` ```
**Output:** array of refs, each: **Output:** `{ refs: Ref[] }` — the MCP protocol requires `structuredContent`
to be a JSON object, so the array of matches is wrapped under `refs`. Each
ref is shaped:
```typescript ```typescript
{ {

View file

@ -0,0 +1,802 @@
# MCP Tool Polish: Slim Research Surface + Spec Compliance
**Date:** 2026-05-16
**Author:** Andrey Avtomonov
**Status:** Design — pending implementation plan
## Background
KTX currently exposes 25 MCP tools across context, semantic-layer, ingest, and
scan ports (`packages/context/src/mcp/context-tools.ts`). The
`ktx-analytics` SKILL.md (`packages/cli/src/skills/analytics/SKILL.md`)
is installed into every supported MCP client by `ktx setup --agents` and
already describes a Douala-equivalent research methodology (Discover → Inspect
→ Resolve → Plan → Query → Validate → Capture) that references nine of those
tools.
A recent in-session audit surfaced three real bug classes:
- **`structuredContent` shape** — `discover_data` returned a bare array; MCP
requires an object. Fixed.
- **Union-shape LLM drift**`sl_query.order_by` accepted
`{ field, direction }` but Claude emitted Cube-style `{ id, desc }`. Fixed
via a `z.preprocess` that normalizes the alt-shape before strict validation.
- **Contract leak to the Python daemon**`compileLocalSlQuery` skipped
`toResolvedWire` and sent TS-only authoring fields (`usage`,
`inherits_columns_from`) to a Pydantic model with `extra="forbid"`. Fixed.
The audit also identified systemic gaps applicable across the surface: no
per-field `.describe()` outside `memory_capture*`, no `outputSchema` declared
anywhere, no MCP tool annotations, lingering union-drift risk on
`slQueryDimensionSchema` (`{ dimension, granularity }`) and
`entityDetailsTableRefSchema` (`{ schema, table }`), and inconsistent error
handling (only `sql_execution` wraps thrown errors in-band per MCP spec; the
other 24 let exceptions propagate as JSON-RPC errors).
The current MCP spec (2025-11-25) provides several mechanisms KTX does not
use: `outputSchema` with `structuredContent` validation, tool annotations
(`readOnlyHint`/`destructiveHint`/`idempotentHint`/`openWorldHint`/`title`),
progress notifications via `_meta.progressToken`, and a clear in-band error
contract (`isError: true` with text content).
The 25-tool surface is also wider than the agent needs. The `ktx-analytics`
SKILL only orchestrates nine of them; the rest are admin/setup/maintenance
operations that are better served by a `ktx`-CLI flow (and a future
`ktx-admin` SKILL — out of scope for this spec).
## Goals
- Reduce the MCP-registered surface to **11 tools** focused on the research
loop: `connection_list`, `discover_data`, `wiki_search`, `wiki_read`,
`entity_details`, `dictionary_search`, `sl_read_source`, `sl_query`,
`sql_execution`, `memory_ingest` (new), `memory_ingest_status` (new). The
remaining 14 tools become CLI-only by removing their MCP registration; their
implementations stay in `packages/context/src/` to back the CLI.
- Replace `memory_capture` / `memory_capture_status` with `memory_ingest` /
`memory_ingest_status`. The new tool takes free-form markdown `content`
plus optional `connectionId`; the memory agent triages into wiki and SL as
before.
- Apply the per-tool polish kit on every retained tool: MCP tool annotations,
`outputSchema`, per-field `.describe()`, rewritten tool descriptions,
standardized in-band error handling, union-drift normalization on the two
remaining at-risk schemas, and a type-narrowed `jsonToolResult`.
- Emit MCP progress notifications from `sql_execution` and `sl_query`.
- Update `ktx-analytics` SKILL.md to use the renamed tool, broaden the capture
step, and document multi-connection routing.
## Non-Goals
- Admin CLI skill (separate spec).
- Deleting the source code of the admin tools (deferred follow-up gated by
the admin CLI skill landing).
- MCP resources (subscribable wiki / SL).
- MCP prompts pushed by the server (the analytics SKILL is the equivalent).
- Elicitation, sampling, tool icons.
- A code-execution tool / Python sandbox (separate spec; the analytics
workflow does not require one for the goals above).
- Per-client schema-feature workarounds beyond what the audit findings
already cover. Codex's no-header-auth limitation is unrelated to tool
shape and is left to `setup-agents.ts` to document.
- Multi-tenancy, telemetry, rate limiting.
## Design
### 1. Surface change
#### 1.1 Retained tools (11)
| # | Tool | Port |
|---|---|---|
| 1 | `connection_list` | `KtxConnectionsMcpPort.list` |
| 2 | `discover_data` | `KtxDiscoverDataMcpPort.search` |
| 3 | `wiki_search` | `KtxKnowledgeMcpPort.search` |
| 4 | `wiki_read` | `KtxKnowledgeMcpPort.read` |
| 5 | `entity_details` | `KtxEntityDetailsMcpPort.read` |
| 6 | `dictionary_search` | `KtxDictionarySearchMcpPort.search` |
| 7 | `sl_read_source` | `KtxSemanticLayerMcpPort.readSource` |
| 8 | `sl_query` | `KtxSemanticLayerMcpPort.query` |
| 9 | `sql_execution` | `KtxSqlExecutionMcpPort.execute` |
| 10 | `memory_ingest` | New port `KtxMemoryIngestMcpPort.ingest` |
| 11 | `memory_ingest_status` | `KtxMemoryIngestMcpPort.status` |
`connection_list` is retained because in multi-connection projects, the agent
needs a way to enumerate available connections before issuing a `sql_execution`
or `sl_query` against a specific one.
**`connectionId` resolution per retained tool** (auto-resolution exists today
only on the local SL path; do not broaden it as part of this spec):
| Tool | `connectionId` | Auto-resolves to single connection if omitted? |
|---|---|---|
| `connection_list` | n/a | n/a |
| `discover_data` | optional | no — search is run unscoped when omitted |
| `wiki_search` | n/a | n/a |
| `wiki_read` | n/a | n/a |
| `entity_details` | required | no |
| `dictionary_search` | optional | no — search is run unscoped when omitted |
| `sl_read_source` | required | no |
| `sl_query` | optional | yes — `resolveLocalConnectionId` (`packages/context/src/sl/local-query.ts`) auto-resolves when the project has exactly one connection |
| `sql_execution` | required | no |
| `memory_ingest` | optional | no — omitted means "global" knowledge (wiki only — see below) |
| `memory_ingest_status` | n/a | n/a |
The skill update in §3 must reflect this matrix: when `connection_list` shows
multiple connections, the agent always passes `connectionId` for the required
tools and for `sl_query`/`discover_data`/`dictionary_search` whenever the user
intent pins a specific warehouse.
**`memory_ingest` connectionId semantics — important constraint.** The
underlying `MemoryAgentService.ingest` derives `hasSL = !!input.connectionId`
(`packages/context/src/memory/memory-agent.service.ts:55`) and only wires the
SL-capable toolset when `connectionId` is supplied
(`packages/context/src/memory/memory-agent.service.ts:116-118`). Therefore
`memory_ingest` can update the semantic layer **only** when `connectionId` is
provided. Omit `connectionId` only for genuinely global wiki-only knowledge
(company-wide policies, vocabulary, user preferences); supply `connectionId`
for any knowledge that touches a specific warehouse — including measure
definitions, schema gotchas, and any wording like "in our warehouse" or "this
warehouse". The §3 SKILL update and the worked example must enforce this.
#### 1.2 Removed from MCP registration
`connection_test`, `wiki_write`, `sl_list_sources`, `sl_write_source`,
`sl_validate`, `ingest_trigger`, `ingest_status`, `ingest_report`,
`ingest_replay`, `scan_trigger`, `scan_status`, `scan_report`,
`scan_list_artifacts`, `scan_read_artifact`,
plus `memory_capture` and `memory_capture_status` (replaced).
The conditional registration blocks in `registerKtxContextTools` for these
ports are removed. The underlying `KtxIngestMcpPort`, `KtxScanMcpPort`, etc.
implementations stay; the `ktx` CLI uses them directly. The `KtxMcpContextPorts`
type drops the removed `ingest?`, `scan?`, etc. fields. `MemoryCapturePort` is
renamed to `MemoryIngestPort`.
#### 1.3 New tool — `memory_ingest`
Replaces `memory_capture`. The change is a rename + a slightly relaxed input
contract; the underlying `MemoryCaptureService`
(`packages/context/src/memory/memory-runs.ts:81`) is reused as-is and renamed to
`MemoryIngestService`. No alias, no migration shim — per the standing
no-back-compat rule, the rename is atomic with the SKILL update.
**Final internal API shape after the rename — no compatibility wrappers:**
| Old name | New name |
|---|---|
| `MemoryCaptureService` (class) | `MemoryIngestService` |
| `MemoryCaptureService.capture(input)` (method) | `MemoryIngestService.ingest(input)` |
| `MemoryCaptureServiceDeps` | `MemoryIngestServiceDeps` |
| `MemoryCaptureStartResult` | `MemoryIngestStartResult` |
| `MemoryCaptureStatus` (return type) | `MemoryIngestStatus` |
| `MemoryCapturePort` (in `mcp/types.ts`) | `MemoryIngestPort` (with `.ingest()` and `.status()`) |
| `MemoryCapturePort.capture()` | `MemoryIngestPort.ingest()` |
| `TextMemoryCapturePort` (CLI, `text-ingest.ts`) | `TextMemoryIngestPort` (with `.ingest()`, `.waitForRun()`, `.status()`) |
| `createLocalProjectMemoryCapture` factory | `createLocalProjectMemoryIngest` |
Every internal call site (`packages/context/src/mcp/server.ts`,
`packages/context/src/mcp/local-project-ports.ts`,
`packages/cli/src/mcp-server-factory.ts`, `packages/cli/src/text-ingest.ts`,
their tests, and the `packages/context/src/memory/index.ts` re-exports) is
updated in lockstep. The agent-facing `MemoryAgentService.ingest` method and
its `MemoryAgentInput` type are unchanged.
**Mapping `memory_ingest` input → `MemoryAgentInput`** (defined in
`packages/context/src/memory/types.ts`):
| `MemoryAgentInput` field | Value supplied by `memory_ingest` handler |
|---|---|
| `userId` | `userContext.userId` (existing pattern) |
| `chatId` | `mcp-${randomUUID()}` (existing pattern) |
| `userMessage` | synthetic framing string, e.g. `Ingest external knowledge into KTX memory.` |
| `assistantMessage` | input.`content` |
| `connectionId` | input.`connectionId` (when provided) |
| `sourceType` | `'external_ingest'` |
The free-form markdown is routed into `assistantMessage` (not `userMessage`)
with a synthetic framing `userMessage`, mirroring the existing CLI text-ingest
path (`packages/cli/src/text-ingest.ts:295-302`). This mapping is required so
that `detectCaptureSignals`
(`packages/context/src/memory/capture-signals.ts:14`) can fire its
`assistantMessage`-keyed cues — SQL aggregates, LookML structure, and
definition tables — for artifact-like content. Routing content into
`userMessage` would lose those signals and silently degrade triage parity with
CLI ingest. The existing memory-agent prompt and tests already expect this
shape; no changes to the memory agent itself are required.
Acceptance criterion: an MCP `memory_ingest` call with the same markdown
content as a CLI `ktx ingest text` invocation must produce identical
`CaptureSignals` (knowledge / sl / dialect / reasons) — covered by a parity
test that feeds the same fixture content through both ingest entry points and
asserts equal `detectCaptureSignals` output.
**Input schema:**
```typescript
const memoryIngestSchema = z.object({
content: z
.string()
.min(1)
.describe(
'Free-form markdown to ingest. Include the knowledge itself plus any ' +
'context (source, the user\'s question, why this came up) that the ' +
'memory agent should consider when triaging into wiki/SL.',
),
connectionId: z
.string()
.min(1)
.optional()
.describe(
'Scope this memory to a specific connection. REQUIRED when the knowledge ' +
'is warehouse-specific (measure definitions, schema gotchas, anything ' +
'tied to a particular warehouse) — without it the memory agent cannot ' +
'update the semantic layer and the knowledge will land as wiki-only. ' +
'Omit only for genuinely global wiki knowledge (company-wide policies, ' +
'vocabulary, user preferences).',
),
});
```
**Tool description:**
> Ingest free-form knowledge into KTX's durable memory so it is available to
> future turns. Call this whenever a research turn produces something worth
> remembering — business rules, metric definitions, gotchas, schema
> explanations, recurring findings — **or** whenever the user asks you to
> remember something. Pass everything in `content` as markdown: the finding,
> plus any source or context that helps the memory agent triage. KTX's memory
> agent decides whether the content belongs in the wiki, the semantic layer,
> or both. Each call is a feedback loop — better notes here mean smarter
> `discover_data` and `wiki_search` results for everyone next time.
**Returns:** `{ runId: string }` — same shape as today's `memory_capture`.
**`memory_ingest_status`** mirrors today's `memory_capture_status` exactly,
renamed only.
### 2. Per-tool polish kit
#### 2.0 Registration topology — memory tools must share the polish path
Today `memory_capture` / `memory_capture_status` are registered in
`packages/context/src/mcp/server.ts` via direct `deps.server.registerTool`
calls (`server.ts:23,45`), **bypassing** `registerParsedTool` in
`context-tools.ts`. If left as-is, the polish kit below
(annotations, `outputSchema`, in-band error wrapping, per-field `.describe()`)
would not apply to `memory_ingest` / `memory_ingest_status`, contradicting the
"all 11 tools" acceptance criteria.
Therefore, as part of the polish-kit PR (PR 2), one of the following must
happen — the implementation plan picks:
1. **Preferred:** Move `memory_ingest` and `memory_ingest_status` registration
into `registerKtxContextTools` so they go through `registerParsedTool` like
every other tool. The `MemoryIngestPort` becomes a `contextTools.memoryIngest`
port and the standalone `registerMemoryCaptureTools` helper in `server.ts`
is deleted.
2. **Acceptable fallback:** Keep `registerMemoryIngestTools` in `server.ts`
but rewrite it to call `registerParsedTool` (exported from
`context-tools.ts`) so the same annotations / `outputSchema` /
error-wrapping plumbing is applied uniformly.
Either way, every checklist item in §§2.12.4 must apply to the two memory
tools, and the §Verification annotations and `outputSchema` tests must cover
them.
#### 2.1 Tool annotations
Every tool gets annotations and a `title`:
| Tool | title | readOnly | destructive | idempotent | openWorld |
|---|---|:--:|:--:|:--:|:--:|
| `connection_list` | Connection List | ✓ | — | ✓ | — |
| `discover_data` | Discover Data | ✓ | — | — | — |
| `wiki_search` | Wiki Search | ✓ | — | — | — |
| `wiki_read` | Wiki Read | ✓ | — | ✓ | — |
| `entity_details` | Entity Details | ✓ | — | ✓ | — |
| `dictionary_search` | Dictionary Search | ✓ | — | — | — |
| `sl_read_source` | Semantic Layer Read Source | ✓ | — | ✓ | — |
| `sl_query` | Semantic Layer Query | ✓ | — | — | — |
| `sql_execution` | SQL Execution | ✓ | — | — | — |
| `memory_ingest` | Memory Ingest | — | ✓ | — | — |
| `memory_ingest_status` | Memory Ingest Status | ✓ | — | omit | — |
`openWorldHint: false` for every tool — even `sql_execution` targets a
configured, bounded warehouse, not the web. `sql_execution` is `readOnlyHint:
true` because the server-side parser enforces read-only (`assertReadOnlySql`).
`destructiveHint` is omitted (defaults to `false`) for read-only tools per the
MCP spec; explicit `false` is fine but redundant.
`ToolAnnotations` are static optional booleans per the MCP 2025-11-25 schema
(`title?`, `readOnlyHint?`, `destructiveHint?`, `idempotentHint?`,
`openWorldHint?` — no state-dependent variants). `idempotentHint` describes
whether repeated calls have additional environmental effect and is most
meaningful when `readOnlyHint` is `false`. For `memory_ingest_status`, which is
a polling read whose response shape changes while a run is active, leave
`idempotentHint` unset — the tool is read-only but not statically idempotent.
`registerTool` accepts annotations in the `config` object today; this is a
plumbing change in `registerParsedTool` to forward them.
#### 2.2 `outputSchema` on all 11 tools
Per the MCP 2025-11-25 spec, clients SHOULD validate `structuredContent`
against `outputSchema` when declared. Authoring is mechanical: each response
shape already typed in `packages/context/src/mcp/types.ts` gets a parallel
Zod schema and is passed as `outputSchema` to `registerTool`.
`registerParsedTool` is extended to accept an optional `outputSchema` arg and
forward it to `server.registerTool`. The Zod schemas live alongside the
input schemas in `context-tools.ts` (or a sibling `tool-output-schemas.ts` if
the file grows too large).
Example for `discover_data`:
```typescript
const discoverDataOutputSchema = z.object({
refs: z.array(
z.object({
kind: discoverDataKindSchema,
id: z.string(),
score: z.number(),
summary: z.string().nullable(),
snippet: z.string().nullable(),
matchedOn: z.enum(['name', 'display', 'description', 'comment', 'expr', 'sample_value', 'body']),
connectionId: z.string().optional(),
tableRef: z.object({ catalog: z.string().nullable(), db: z.string().nullable(), name: z.string() }).optional(),
columnName: z.string().optional(),
}),
),
});
```
#### 2.3 Per-field `.describe()` on every input
Anthropic's documented mechanism for fighting model drift, already used in
`memory_capture*`. Applied to every input field on every retained tool.
Highest leverage: `sl_query`, `entity_details`, `dictionary_search`,
`sql_execution`, `memory_ingest`. Tool-level `description` strings are
rewritten to be longer with one concrete example shape inlined (the technique
that fixed `order_by` model drift in this session).
#### 2.4 In-band error wrapping in `registerParsedTool`
Per MCP spec, tools return handler/runtime errors as `isError: true` + text
content, not JSON-RPC errors. Move the try/catch into the `registerParsedTool`
helper so every tool consistently surfaces handler exceptions as
`jsonErrorToolResult`. `sql_execution`'s local try/catch is removed (the
helper handles it).
**Scope — what becomes in-band vs. what stays JSON-RPC.** The MCP SDK
pre-validates incoming arguments against the registered `inputSchema` before
the tool callback runs, and surfaces validation failures as
`McpError(InvalidParams)` / JSON-RPC errors
(`@modelcontextprotocol/sdk/dist/esm/server/mcp.js` `validateToolInput`,
~line 166). KTX cannot intercept those without forking the SDK and we will not.
Therefore:
- Schema-validation failures on input → remain JSON-RPC `InvalidParams` errors,
emitted by the SDK before our handler runs. This is the documented MCP
behavior; clients already handle it.
- Handler exceptions, port/driver errors, and any post-validation runtime
errors thrown inside the tool body → wrapped in-band as
`{ isError: true, content: [{ type: 'text', ... }] }` by
`registerParsedTool`'s catch.
- The redundant `inputSchema.parse(input)` inside `registerParsedTool` may be
kept as defense-in-depth (e.g., for the rare path where the SDK was given a
raw shape and a downstream change loosens validation) or removed; either is
acceptable. If kept, parse failures here are wrapped in-band as well, but in
practice they are unreachable for valid SDK registrations because the SDK
has already parsed against the same schema.
```typescript
function registerParsedTool<TInput extends z.ZodType, TOutput extends z.ZodType>(
server: KtxMcpServerLike,
name: string,
config: { title: string; description: string; inputSchema: unknown; outputSchema?: unknown; annotations: ToolAnnotations },
inputSchema: TInput,
handler: (input: z.infer<TInput>) => Promise<KtxMcpToolResult>,
outputSchema?: TOutput,
): void {
server.registerTool(name, { ...config, outputSchema: outputSchema ? outputSchema : undefined }, async (input) => {
try {
return await handler(inputSchema.parse(input));
} catch (error) {
return jsonErrorToolResult(formatToolError(error));
}
});
}
```
A small `formatToolError` helper renders Zod errors with `path: message` lines
and falls through to `error.message` / `String(error)` for non-Zod cases.
Acceptance tests in §Verification must therefore split the error path:
- Bad input shape (rejected by SDK pre-validation) → expect a thrown
`McpError`/`InvalidParams` JSON-RPC error, not `isError: true`.
- Handler-thrown / port-thrown error (e.g., unknown `connectionId`, driver
failure) → expect `{ isError: true, content: [{ type: 'text', ... }] }`.
#### 2.5 Union-drift normalization
Apply the same `z.preprocess` pattern used for `order_by` to the two remaining
at-risk unions:
**`slQueryDimensionSchema`** — accept `{ dimension, granularity }` (Cube
convention) as an alias for `{ field, granularity }`. Bare strings continue to
work unchanged.
```typescript
const slQueryDimensionSchema = z.preprocess(
(value) => {
if (typeof value === 'string') return { field: value };
if (value && typeof value === 'object' && !Array.isArray(value)) {
const obj = { ...(value as Record<string, unknown>) };
if (!('field' in obj) && typeof obj.dimension === 'string') obj.field = obj.dimension;
return obj;
}
return value;
},
z.object({
field: z.string().min(1).describe('Dimension to group by, e.g. "orders.created_at" or a SL dimension key.'),
granularity: z.string().min(1).optional().describe('Time granularity for time dimensions: day, week, month, quarter, year.'),
}),
);
```
**`entityDetailsTableRefSchema`** — accept `{ schema, table }` (BigQuery /
SQL-style convention) as an alias for `{ db, name }`. Today's schema requires
`catalog`, `db`, and `name` (`packages/context/src/mcp/context-tools.ts:169`),
so the alias path must also default `catalog` to `null` to satisfy the
validator. Either of the two equivalent shapes below is acceptable; the
implementation plan picks one:
1. Make `catalog` (and `db`) `.nullable().default(null)` so the alias path
doesn't have to set them, and bare `{ schema, table }` is accepted with
`catalog === null`, `db === schema`, `name === table`.
2. Have the preprocess unconditionally fill missing `catalog`/`db` with `null`.
Acceptance criterion: a tool call with `{ table: { schema: "public", table: "orders" } }` parses successfully and resolves to `{ catalog: null, db: "public", name: "orders" }`. Existing callers passing `{ catalog, db, name }` continue to work unchanged.
```typescript
const entityDetailsTableRefSchema = z.preprocess(
(value) => {
if (value && typeof value === 'object' && !Array.isArray(value)) {
const obj = { ...(value as Record<string, unknown>) };
if (!('db' in obj) && typeof obj.schema === 'string') obj.db = obj.schema;
if (!('name' in obj) && typeof obj.table === 'string') obj.name = obj.table;
if (!('catalog' in obj)) obj.catalog = null;
return obj;
}
return value;
},
z.object({
catalog: z.string().nullable().describe('Catalog/project (BigQuery project, Snowflake database). null when not applicable.'),
db: z.string().nullable().describe('Schema/database for the table. null when not applicable.'),
name: z.string().min(1).describe('Table name.'),
}),
);
```
`slQueryMeasureSchema` is **not** changed — bare strings cover the common case
and `{ expr, name }` matches Cube's measure-with-alias convention; no observed
drift.
#### 2.6 Type-narrow `jsonToolResult`
The goal is to forbid arrays at compile time without breaking the existing
`interface`-typed response shapes (e.g. `KtxKnowledgeSearchResponse` in
`packages/context/src/mcp/types.ts`). `Record<string, unknown>` is **not**
acceptable as the constraint because TypeScript interfaces without an index
signature are not assignable to it; using it would cause widespread compile
failures on valid object responses.
Use a non-array object constraint instead, e.g.:
```typescript
type NonArrayObject = object & { length?: never };
export function jsonToolResult<T extends NonArrayObject>(
structuredContent: T,
): KtxMcpToolResult<T> { ... }
```
Acceptance criteria:
- A bare array literal at a call site fails to type-check (catches the
`discover_data` bug class).
- Every existing `interface`-typed response (`KtxKnowledgeSearchResponse`,
`KtxSemanticLayerQueryResponse`, `KtxSqlExecutionResponse`,
`KtxEntityDetailsResponse`, `KtxDictionarySearchResponse`,
`KtxDiscoverDataResponse`, `KtxConnectionTestResponse`,
`KtxSemanticLayerReadResponse`, `KtxSemanticLayerListResponse`,
`KtxKnowledgePage`, memory capture/status response shapes) continues to
type-check at every existing `jsonToolResult` call site without modification.
Implementation plan can substitute an equivalent narrowing if it is more
idiomatic; the contract is "no arrays, no breaking interface assignability."
Pure defensive type change; no runtime effect on current code.
#### 2.7 Enforce the `toResolvedWire` invariant
Add a doc comment on `KtxSemanticLayerComputePort.query` and
`.validateSources` stating that callers must pass `toResolvedWire`-sanitized
sources to prevent the daemon `usage`-leak bug from regressing if a new code
path bypasses `SemanticLayerService`.
The doc comment alone is not sufficient — there is already an unsanitized
caller. `loadComputableSources` in
`packages/context/src/mcp/local-project-ports.ts` (~line 311) parses YAML and
pushes the raw record into `validateSources`
(`packages/context/src/mcp/local-project-ports.ts:586`). It must be brought into
conformance by sanitizing each record with `toResolvedWire` before handing it
to `validateSources`, mirroring the existing sanitization in
`packages/context/src/sl/local-query.ts:76`. Acceptance criterion: every
`KtxSemanticLayerComputePort.query` and `.validateSources` call site in the
repo passes `toResolvedWire`-sanitized records, verified by code review and
covered by the existing `local-query.test.ts` / `local-project-ports.test.ts`
tests for the relevant paths. Note that `sl_validate` is removed from MCP
registration (§1.2) but the underlying port keeps backing the CLI, so the
invariant must hold for the CLI path too.
#### 2.8 Progress notifications — `sql_execution` + `sl_query`
Per MCP spec, the caller may include `params._meta.progressToken` in the
request; the server emits `notifications/progress` with `{ progressToken,
progress, total?, message }` at stage transitions.
`KtxSqlExecutionMcpPort.execute` and `KtxSemanticLayerMcpPort.query` are
extended with an optional `onProgress?: (event: { progress: number; total?: number; message: string }) => void` parameter. The MCP tool handlers wire
`onProgress` to the SDK's notification channel via the handler-context object
passed by `server.registerTool`. Non-progress-supporting clients ignore the
events.
Emitted stages:
- `sql_execution`: `"Validating SQL"` (progress 0.0) → `"Executing"` (0.3) → `"Fetched N rows"` (1.0).
- `sl_query`: `"Compiling query"` (0.0) → `"Generating SQL"` (0.3) → `"Executing"` (0.6) → `"Fetched N rows"` (1.0).
Progress emission is best-effort; if the underlying port can't report a stage
boundary (e.g., a driver doesn't expose progress callbacks), the stage is
simply skipped.
`memory_ingest` does not emit progress — `runId` + `memory_ingest_status`
polling is the documented async pattern for it.
### 3. `ktx-analytics` SKILL.md refinements
File: `packages/cli/src/skills/analytics/SKILL.md`.
**Step 7 rewrite** (current vs. new):
Current:
> 7. **Capture durable learnings** - at the end of the turn, call `memory_capture` when the investigation produced reusable business context, metric definitions, or schema knowledge.
New:
> 7. **Capture durable learnings** - call `memory_ingest` whenever a turn produces something worth remembering (business rules, metric definitions, schema gotchas, recurring findings) **or** whenever the user asks you to remember something. Pass markdown in `content` including any source context the memory agent should weigh. Each call is a feedback loop — better notes today mean smarter `discover_data` and `wiki_search` results tomorrow.
**Tool-name updates** throughout: every `memory_capture` reference becomes
`memory_ingest`.
**Multi-connection rule** added under `<rules>` — phrased to match the §1.1
connection matrix so the agent does not over-scope unscoped tools:
> When `connection_list` shows multiple connections, pass an explicit
> `connectionId` to every tool that takes one **and where user intent pins a
> specific warehouse**. The matrix is:
>
> - **Required:** `entity_details`, `sl_read_source`, `sql_execution`.
> - **Required when user intent is warehouse-specific (including any wording
> like "in our warehouse" / "this warehouse"):** `memory_ingest` — without
> `connectionId`, the memory agent cannot update the semantic layer and the
> knowledge will land as wiki-only.
> - **Pass when intent pins a warehouse, otherwise omit for unscoped
> discovery:** `sl_query`, `discover_data`, `dictionary_search`.
> - **Never pass `connectionId` (the tool does not accept one):**
> `connection_list`, `wiki_search`, `wiki_read`, `memory_ingest_status`.
>
> If intent is ambiguous for a required-or-scoped tool, ask the user which
> warehouse before calling — do not guess.
**One new worked example** demonstrating user-driven ingest:
> **Input:** "Heads up — ARR is always reported in cents in our warehouse."
>
> **Workflow:**
> 1. If multiple connections, call `connection_list` and pick the warehouse the
> user means (asking if ambiguous). Pass its id as `connectionId` so the
> memory agent can update the semantic layer, not just the wiki.
> 2. `memory_ingest({ connectionId: "<warehouse-id>", content: "ARR is reported in cents (not dollars) in this warehouse. Multiply by 0.01 for dollar amounts. Source: user clarification." })` — no analysis turn; just remember.
The existing Discover → Inspect → Resolve → Plan → Query → Validate → Capture
workflow stays. The existing two examples are updated only to reflect the
`memory_capture``memory_ingest` rename.
## Migration / sequencing
Three landings, each independently mergeable, in this order:
### PR 1 — Surface change (atomic)
- Remove the 14 admin tools from `registerKtxContextTools` (conditional
registration blocks deleted).
- Rename `memory_capture``memory_ingest`, `memory_capture_status`
`memory_ingest_status`. Rename `MemoryCapturePort``MemoryIngestPort`,
`MemoryCaptureService``MemoryIngestService`. Update the new tool's input
contract per §1.3.
- Update `packages/context/src/mcp/local-project-ports.ts` and
`packages/context/src/mcp/server.ts` to reflect the renames and the dropped
ports.
- Update `packages/cli/src/skills/analytics/SKILL.md` per §3 in the same
diff.
- Update all tests for the removed/renamed tools.
- Update `docs-site/content/docs/integrations/agent-clients.mdx` to replace
the existing "memory capture" wording (currently at line ~90) with
"memory ingest". This update is unconditional — the file already names the
tool family.
- Update `packages/cli/src/mcp-server-factory.ts` and
`packages/cli/src/text-ingest.ts` (plus their tests) to reflect the
`MemoryCapture*``MemoryIngest*` rename. Both files import
`createLocalProjectMemoryCapture` and use a `memoryCapture` variable, so the
rename does cross the CLI boundary even though the tool surface used by the
CLI is unchanged. Re-export rename in `packages/context/src/memory/index.ts`
is part of this PR.
The CLI's runtime use of the removed admin-tool implementations is unchanged —
only the memory rename touches CLI code.
### PR 2 — Polish kit
Touches all 11 retained tools. Can be one PR or split per family
(annotations + outputSchema + descriptions + error wrapping + union-drift
fixes + `jsonToolResult` type narrowing + `toResolvedWire` doc comment).
### PR 3 — Progress notifications
Extends `KtxSqlExecutionMcpPort` and `KtxSemanticLayerMcpPort` with optional
`onProgress`, wires the MCP handler context's notification channel, emits at
the stage boundaries listed in §2.8.
Eventual **deletion** of the 14 admin tool implementations is a separate
follow-up spec gated on the `ktx-admin` SKILL landing. Until then they remain
in `packages/context/src/` and are used only by the CLI.
## Verification
### Unit tests per retained tool
For each of the 11 retained tools:
- Input schema accepts canonical input.
- Input schema accepts each documented normalized alt-shape (Cube
`{ dimension, granularity }`, `{ schema, table }`, bare-string `order_by`,
Cube-style `{ id, desc }` `order_by`).
- Output schema accepts the response shape returned by the underlying port.
- Error path returns `{ isError: true, content: [{ type: 'text', ... }] }`
(not a thrown exception).
### Schema snapshot test
A `tools/list` snapshot test in `packages/context/src/mcp/server.test.ts`
captures the exact JSON Schema each client receives for every tool. Re-runs
across PRs catch accidental schema drift (e.g., a Zod change silently
broadening the contract).
### Annotations test
Assert every tool's registered config carries the expected `readOnlyHint`,
`destructiveHint`, `idempotentHint`, `openWorldHint`, and `title` per §2.1.
### Multi-client end-to-end smoke
Stdio (Claude Desktop) and Streamable HTTP (Claude Code) are the two
transports; the other four clients (Codex, Cursor, OpenCode, universal) share
one of these transports and their config files are static. Spin up `ktx mcp
stdio` and `ktx mcp start`, call each retained tool through both transports,
verify response shape against `outputSchema`.
### Required commands
The named test files include slow tests that are excluded from the default
`@ktx/context` `test` script and live in `test:slow`
(`packages/context/package.json:127-128`). Implementation must run both:
```bash
pnpm --filter @ktx/context run test
pnpm --filter @ktx/context run test:slow
pnpm --filter @ktx/context run type-check
pnpm --filter @ktx/cli run type-check
pnpm --filter @ktx/cli run test
pnpm run dead-code
```
CLI checks are required because PR 1 renames cross the CLI boundary
(`packages/cli/src/mcp-server-factory.ts`, `packages/cli/src/text-ingest.ts`,
the `MemoryCapture*` re-exports, plus their vitest specs).
`pnpm --filter @ktx/cli run test:slow` should also be added if PR 1 ends up
touching any of the slow-test files enumerated in `packages/cli/package.json`
(`scan.test.ts`, `setup*.test.ts`, etc.); the rename diff today does not, but
the implementation plan must re-check before merging.
When `docs-site/content/docs/integrations/agent-clients.mdx` is touched, also
run the docs-site scripts declared in `docs-site/package.json` — there is no
`lint` script:
```bash
pnpm --filter ktx-docs run build
pnpm --filter ktx-docs run test
```
### Red-green regression
For the union-drift fixes (§2.5): revert the preprocess in `local-query.ts` or
`context-tools.ts`, run the alt-shape test → expect failure, restore →
expect pass. Same pattern as the `order_by` and `usage`-leak fixes in this
session.
## Risks
- **Removed tools surprise a user who depended on them via MCP.** Mitigated
by the no-back-compat rule (KTX is pre-public) and by the SKILL update
landing atomically with the surface change. Users on `ktx setup --agents`
flow get the updated SKILL the next time they re-run setup.
- **`outputSchema` validation breaks a client that doesn't tolerate
unrecognized JSON Schema keywords.** Mitigated by emitting `outputSchema`
via the same `server.registerTool` path that already produces `inputSchema`,
so both schemas are serialized by the MCP SDK as JSON Schema 2020-12 (the
dialect the SDK's tool-list types declare —
`@modelcontextprotocol/sdk/.../types.js` `inputSchema` / `outputSchema`
fields, and Appendix B). The spec uses SHOULD-validate semantics, not MUST.
The snapshot test catches drift; the multi-client smoke confirms
compatibility.
- **Progress notifications increase notification volume for clients that
poll.** Mitigated by stage-based emission (3-4 events per call max). Clients
that don't support progress simply ignore the events.
- **Renaming `MemoryCapturePort`/`MemoryCaptureService` cascades through
internal callers.** Cascade is bounded to `packages/context/src/memory/`,
`packages/context/src/mcp/`, and their tests; type-checker catches missed
call sites.
## Open Questions
None at design time. Open items for the implementation plan:
- Final wording for the rewritten tool descriptions on `discover_data`,
`entity_details`, `dictionary_search`, `sl_read_source`, `sl_query`,
`sql_execution`. (Drafts can be authored during PR 2.)
- Whether `formatToolError` should redact path elements for security
(probably not — these are local-only MCP servers, and the existing error
shape doesn't redact).
- Whether to split PR 2 into per-family sub-PRs or keep it monolithic. Default
is monolithic since the polish-kit changes touch the same files and tests.
## Appendix A — File map
| Change | File |
|---|---|
| Tool registration removed (14 tools) | `packages/context/src/mcp/context-tools.ts` |
| Memory rename + route memory_ingest tools through the shared polish path (§2.0) | `packages/context/src/mcp/server.ts`, `packages/context/src/mcp/context-tools.ts`, `packages/context/src/memory/*` |
| Local ports update | `packages/context/src/mcp/local-project-ports.ts` |
| Port types update | `packages/context/src/mcp/types.ts` |
| Annotations, outputSchema, describe, error wrapping, union preprocess | `packages/context/src/mcp/context-tools.ts` |
| `jsonToolResult` type narrowing | `packages/context/src/mcp/context-tools.ts` |
| `toResolvedWire` invariant comment | `packages/context/src/daemon/semantic-layer-compute.ts` |
| Progress callback plumbing | `packages/context/src/daemon/semantic-layer-compute.ts`, `packages/context/src/sl/local-query.ts`, `packages/context/src/mcp/local-project-ports.ts` (`executeValidatedReadOnlySql`) |
| Tests | `packages/context/src/mcp/server.test.ts`, `packages/context/src/mcp/local-project-ports.test.ts`, `packages/context/src/sl/local-query.test.ts` |
| Skill | `packages/cli/src/skills/analytics/SKILL.md` |
| Docs | `docs-site/content/docs/integrations/agent-clients.mdx` |
## Appendix B — MCP-spec cross-reference
- Tool annotations (`readOnlyHint`, `destructiveHint`, `idempotentHint`,
`openWorldHint`, `title`): MCP 2025-11-25 spec `/schema` "ToolAnnotations".
- `outputSchema` with `structuredContent` SHOULD-validate: spec
`/server/tools` "Tool Result > Structured Content".
- In-band error contract (`isError: true` + text content, not JSON-RPC
error): spec `/server/tools` "Tool Execution Error Example".
- Progress notifications via `params._meta.progressToken`
`notifications/progress`: spec `/basic/utilities/progress`.
- `inputSchema.type: "object"` requirement and JSON Schema 2020-12 default
dialect: spec `/server/tools` "Tool > inputSchema" and SEP 1613.

View file

@ -48,6 +48,7 @@
"@ktx/llm": "workspace:*", "@ktx/llm": "workspace:*",
"@modelcontextprotocol/sdk": "^1.29.0", "@modelcontextprotocol/sdk": "^1.29.0",
"commander": "14.0.3", "commander": "14.0.3",
"fflate": "^0.8.2",
"ink": "^7.0.2", "ink": "^7.0.2",
"react": "^19.2.6", "react": "^19.2.6",
"zod": "^4.4.3" "zod": "^4.4.3"

View file

@ -186,6 +186,10 @@ function shouldSuppressProjectDirLine(path: string[], options: Record<string, un
return true; return true;
} }
if (commandPathKey === 'ktx mcp stdio') {
return true;
}
if ( if (
commandPathKey === 'ktx status' && commandPathKey === 'ktx status' &&
typeof options.projectDir !== 'string' && typeof options.projectDir !== 'string' &&

View file

@ -39,6 +39,7 @@ export interface KtxCliDeps {
stopDaemon?: typeof import('./managed-mcp-daemon.js').stopKtxMcpDaemon; stopDaemon?: typeof import('./managed-mcp-daemon.js').stopKtxMcpDaemon;
readStatus?: typeof import('./managed-mcp-daemon.js').readKtxMcpDaemonStatus; readStatus?: typeof import('./managed-mcp-daemon.js').readKtxMcpDaemonStatus;
runServer?: typeof import('./mcp-http-server.js').runKtxMcpHttpServer; runServer?: typeof import('./mcp-http-server.js').runKtxMcpHttpServer;
runStdioServer?: typeof import('./mcp-stdio-server.js').runKtxMcpStdioServer;
}; };
} }

View file

@ -35,6 +35,7 @@ describe('registerMcpCommands', () => {
'serve-internal', 'serve-internal',
'start', 'start',
'status', 'status',
'stdio',
'stop', 'stop',
]); ]);
expect( expect(
@ -54,4 +55,48 @@ describe('registerMcpCommands', () => {
); );
expect(startDaemon).not.toHaveBeenCalled(); expect(startDaemon).not.toHaveBeenCalled();
}); });
it('prints "already running" when startDaemon reports already-running', async () => {
const program = new Command().exitOverride().option('--project-dir <path>');
const startDaemon = vi.fn().mockResolvedValue({
status: 'already-running',
url: 'http://127.0.0.1:7878/mcp',
state: {
schemaVersion: 1,
pid: 4242,
host: '127.0.0.1',
port: 7878,
tokenAuth: false,
projectDir: '/tmp/ktx-already',
startedAt: '2026-05-14T00:00:00.000Z',
logPath: '/tmp/ktx-already/.ktx/logs/mcp.log',
},
});
const context = makeContext({ deps: { mcp: { startDaemon } } });
registerMcpCommands(program, context);
await program.parseAsync(['--project-dir', '/tmp/ktx-already', 'mcp', 'start'], { from: 'user' });
expect(startDaemon).toHaveBeenCalledTimes(1);
expect(context.io.stdout.write).toHaveBeenCalledWith(
'KTX MCP daemon already running: http://127.0.0.1:7878/mcp\n',
);
});
it('runs the stdio server with the resolved project directory', async () => {
const program = new Command().exitOverride().option('--project-dir <path>');
const runStdioServer = vi.fn().mockResolvedValue(undefined);
const context = makeContext({ deps: { mcp: { runStdioServer } } });
registerMcpCommands(program, context);
await expect(program.parseAsync(['--project-dir', '/tmp/ktx6', 'mcp', 'stdio'], { from: 'user' })).resolves.toBe(
program,
);
expect(runStdioServer).toHaveBeenCalledWith({
projectDir: '/tmp/ktx6',
cliVersion: '0.0.0-test',
io: context.io,
});
});
}); });

View file

@ -15,6 +15,7 @@ import {
stopKtxMcpDaemon, stopKtxMcpDaemon,
} from '../managed-mcp-daemon.js'; } from '../managed-mcp-daemon.js';
import { buildMcpSecurityConfig, runKtxMcpHttpServer } from '../mcp-http-server.js'; import { buildMcpSecurityConfig, runKtxMcpHttpServer } from '../mcp-http-server.js';
import { runKtxMcpStdioServer } from '../mcp-stdio-server.js';
function tokenFromOption(value: string | undefined): string | undefined { function tokenFromOption(value: string | undefined): string | undefined {
return value ?? process.env.KTX_MCP_TOKEN; return value ?? process.env.KTX_MCP_TOKEN;
@ -27,6 +28,17 @@ function binPath(): string {
export function registerMcpCommands(program: Command, context: KtxCliCommandContext): void { export function registerMcpCommands(program: Command, context: KtxCliCommandContext): void {
const mcp = program.command('mcp').description('Run the KTX MCP HTTP server'); const mcp = program.command('mcp').description('Run the KTX MCP HTTP server');
mcp
.command('stdio')
.description('Run the KTX MCP server over stdio')
.action(async (_options, command) => {
await (context.deps.mcp?.runStdioServer ?? runKtxMcpStdioServer)({
projectDir: resolveCommandProjectDir(command),
cliVersion: context.packageInfo.version,
io: context.io,
});
});
mcp mcp
.command('start') .command('start')
.description('Start the KTX MCP HTTP server') .description('Start the KTX MCP HTTP server')
@ -70,7 +82,11 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
allowedOrigins: options.allowedOrigin, allowedOrigins: options.allowedOrigin,
binPath: binPath(), binPath: binPath(),
}); });
context.io.stdout.write(`KTX MCP daemon started: ${result.url}\n`); context.io.stdout.write(
result.status === 'started'
? `KTX MCP daemon started: ${result.url}\n`
: `KTX MCP daemon already running: ${result.url}\n`,
);
}); });
mcp mcp

View file

@ -220,6 +220,7 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
.addOption( .addOption(
new Option('--target <target>', 'Agent target').choices([ new Option('--target <target>', 'Agent target').choices([
'claude-code', 'claude-code',
'claude-desktop',
'codex', 'codex',
'cursor', 'cursor',
'opencode', 'opencode',

View file

@ -994,6 +994,37 @@ describe('runContextBuild', () => {
); );
}); });
it('threads the original runtime IO into captured target execution', async () => {
const io = makeIo({ isTTY: true });
const project = projectWithConnections({
warehouse: { driver: 'postgres', context: { queryHistory: { enabled: true } } },
});
const executeTarget = vi.fn(async (target) => successResult(target.connectionId, target.driver, target.operation));
await runContextBuild(
project,
{
projectDir: '/tmp/project',
inputMode: 'auto',
cliVersion: '0.2.0',
runtimeInstallPolicy: 'auto',
},
io.io,
{ executeTarget, now: () => 1000 },
);
expect(executeTarget).toHaveBeenCalledWith(
expect.objectContaining({ connectionId: 'warehouse' }),
expect.objectContaining({ runtimeInstallPolicy: 'auto' }),
expect.objectContaining({
stdout: expect.objectContaining({ isTTY: false }),
}),
expect.objectContaining({
runtimeIo: io.io,
}),
);
});
it('calls onSourceProgress when sources start and finish', async () => { it('calls onSourceProgress when sources start and finish', async () => {
const io = makeIo(); const io = makeIo();
const project = projectWithConnections({ const project = projectWithConnections({

View file

@ -1022,6 +1022,7 @@ export async function runContextBuild(
const progressDeps: KtxPublicIngestDeps = { const progressDeps: KtxPublicIngestDeps = {
scanProgress: createContextBuildProgressPort(updateSchemaPhase), scanProgress: createContextBuildProgressPort(updateSchemaPhase),
ingestProgress: updateIngestPhase, ingestProgress: updateIngestPhase,
runtimeIo: io,
onPhaseStart, onPhaseStart,
onPhaseEnd, onPhaseEnd,
}; };

View file

@ -11,7 +11,7 @@ import {
} from '@ktx/context/ingest'; } from '@ktx/context/ingest';
import { initKtxProject, ktxLocalStateDbPath, loadKtxProject } from '@ktx/context/project'; import { initKtxProject, ktxLocalStateDbPath, loadKtxProject } from '@ktx/context/project';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { type KtxIngestArgs, runKtxIngest } from './ingest.js'; import { type KtxIngestArgs, type KtxIngestDeps, runKtxIngest } from './ingest.js';
import type { KtxCliLocalIngestAdaptersOptions } from './local-adapters.js'; import type { KtxCliLocalIngestAdaptersOptions } from './local-adapters.js';
import { import {
CliLookerSlWritingAgentRunner, CliLookerSlWritingAgentRunner,
@ -1110,6 +1110,7 @@ describe('runKtxIngest', () => {
completedLocalBundleRun(input, input.jobId ?? 'local-job-1'), completedLocalBundleRun(input, input.jobId ?? 'local-job-1'),
); );
const io = makeIo(); const io = makeIo();
const runtimeIo = makeIo({ isTTY: true });
await expect( await expect(
runKtxIngest( runKtxIngest(
@ -1127,6 +1128,9 @@ describe('runKtxIngest', () => {
createAdapters, createAdapters,
runLocalIngest: runLocal, runLocalIngest: runLocal,
jobIdFactory: () => 'local-job-1', jobIdFactory: () => 'local-job-1',
runtimeIo: runtimeIo.io,
} as KtxIngestDeps & {
runtimeIo: typeof runtimeIo.io;
}, },
), ),
).resolves.toBe(0); ).resolves.toBe(0);
@ -1135,7 +1139,7 @@ describe('runKtxIngest', () => {
cliVersion: '0.2.0', cliVersion: '0.2.0',
projectDir, projectDir,
installPolicy: 'auto', installPolicy: 'auto',
io: io.io, io: runtimeIo.io,
}; };
expect(createAdapters).toHaveBeenCalledWith( expect(createAdapters).toHaveBeenCalledWith(
expect.objectContaining({ projectDir }), expect.objectContaining({ projectDir }),

View file

@ -97,6 +97,7 @@ export interface KtxIngestDeps {
| 'pullConfigOptions' | 'pullConfigOptions'
>; >;
progress?: (update: KtxIngestProgressUpdate) => void; progress?: (update: KtxIngestProgressUpdate) => void;
runtimeIo?: KtxIngestIo;
} }
function reportStatus(report: IngestReportSnapshot): 'done' | 'error' { function reportStatus(report: IngestReportSnapshot): 'done' | 'error' {
@ -615,7 +616,7 @@ export async function runKtxIngest(
(deps.runLocalIngest || deps.runLocalMetabaseIngest ? () => [] : createKtxCliLocalIngestAdapters); (deps.runLocalIngest || deps.runLocalMetabaseIngest ? () => [] : createKtxCliLocalIngestAdapters);
const executeLocalIngest = deps.runLocalIngest ?? runLocalIngest; const executeLocalIngest = deps.runLocalIngest ?? runLocalIngest;
const localIngestOptions = deps.localIngestOptions ?? {}; const localIngestOptions = deps.localIngestOptions ?? {};
const managedDaemon = managedDaemonOptionsForIngestRun(args, io); const managedDaemon = managedDaemonOptionsForIngestRun(args, deps.runtimeIo ?? io);
const operationalLogger = createCliOperationalLogger(io, args.outputMode); const operationalLogger = createCliOperationalLogger(io, args.outputMode);
const adapterOptions = { const adapterOptions = {
...(localIngestOptions.pullConfigOptions ?? {}), ...(localIngestOptions.pullConfigOptions ?? {}),

View file

@ -94,6 +94,78 @@ describe('managed MCP daemon lifecycle', () => {
); );
}); });
it('returns already-running without spawning when the daemon is alive at the same host/port', async () => {
await mkdir(join(projectDir, '.ktx'), { recursive: true });
await writeFile(join(projectDir, '.ktx/mcp.json'), `${JSON.stringify(state(projectDir), null, 2)}\n`);
const spawnDaemon = vi.fn(() => child(9999));
const result = await startKtxMcpDaemon({
projectDir,
cliVersion: '0.0.0-test',
host: '127.0.0.1',
port: 7878,
allowedHosts: [],
allowedOrigins: [],
binPath: '/repo/packages/cli/dist/bin.js',
spawnDaemon,
processAlive: vi.fn(() => true),
portAvailable: vi.fn(async () => true),
});
expect(result.status).toBe('already-running');
expect(result.url).toBe('http://127.0.0.1:7878/mcp');
expect(result.state.pid).toBe(4242);
expect(spawnDaemon).not.toHaveBeenCalled();
});
it('throws when the recorded daemon uses a different host or port', async () => {
await mkdir(join(projectDir, '.ktx'), { recursive: true });
await writeFile(join(projectDir, '.ktx/mcp.json'), `${JSON.stringify(state(projectDir), null, 2)}\n`);
const spawnDaemon = vi.fn(() => child(9999));
await expect(
startKtxMcpDaemon({
projectDir,
cliVersion: '0.0.0-test',
host: '127.0.0.1',
port: 9000,
allowedHosts: [],
allowedOrigins: [],
binPath: '/repo/packages/cli/dist/bin.js',
spawnDaemon,
processAlive: vi.fn(() => true),
portAvailable: vi.fn(async () => true),
}),
).rejects.toThrow(/different configuration[\s\S]*ktx mcp stop/);
expect(spawnDaemon).not.toHaveBeenCalled();
});
it('throws when token-auth presence differs from the recorded daemon', async () => {
await mkdir(join(projectDir, '.ktx'), { recursive: true });
await writeFile(
join(projectDir, '.ktx/mcp.json'),
`${JSON.stringify(state(projectDir, { tokenAuth: false }), null, 2)}\n`,
);
const spawnDaemon = vi.fn(() => child(9999));
await expect(
startKtxMcpDaemon({
projectDir,
cliVersion: '0.0.0-test',
host: '127.0.0.1',
port: 7878,
token: 'secret-token',
allowedHosts: [],
allowedOrigins: [],
binPath: '/repo/packages/cli/dist/bin.js',
spawnDaemon,
processAlive: vi.fn(() => true),
portAvailable: vi.fn(async () => true),
}),
).rejects.toThrow(/different configuration/);
expect(spawnDaemon).not.toHaveBeenCalled();
});
it('reports running when the process is alive and health passes', async () => { it('reports running when the process is alive and health passes', async () => {
await mkdir(join(projectDir, '.ktx'), { recursive: true }); await mkdir(join(projectDir, '.ktx'), { recursive: true });
await writeFile(join(projectDir, '.ktx/mcp.json'), `${JSON.stringify(state(projectDir), null, 2)}\n`); await writeFile(join(projectDir, '.ktx/mcp.json'), `${JSON.stringify(state(projectDir), null, 2)}\n`);

View file

@ -121,11 +121,25 @@ export async function startKtxMcpDaemon(options: {
portAvailable?: (host: string, port: number) => Promise<boolean>; portAvailable?: (host: string, port: number) => Promise<boolean>;
spawnDaemon?: typeof defaultSpawnDaemon; spawnDaemon?: typeof defaultSpawnDaemon;
now?: () => Date; now?: () => Date;
}): Promise<{ status: 'started'; state: KtxMcpDaemonState; url: string }> { }): Promise<{ status: 'started' | 'already-running'; state: KtxMcpDaemonState; url: string }> {
const existing = await readState(options.projectDir).catch(() => undefined); const existing = await readState(options.projectDir).catch(() => undefined);
const processAlive = options.processAlive ?? defaultProcessAlive; const processAlive = options.processAlive ?? defaultProcessAlive;
if (existing && processAlive(existing.pid)) { if (existing && processAlive(existing.pid)) {
throw new Error(`KTX MCP daemon is already recorded at http://${existing.host}:${existing.port}/mcp`); const sameConfig =
existing.host === options.host &&
existing.port === options.port &&
existing.tokenAuth === Boolean(options.token);
if (sameConfig) {
return {
status: 'already-running',
state: existing,
url: `http://${existing.host}:${existing.port}/mcp`,
};
}
throw new Error(
`KTX MCP daemon is already running at http://${existing.host}:${existing.port}/mcp ` +
'with a different configuration. Run `ktx mcp stop` first, then start again.',
);
} }
const portAvailable = options.portAvailable ?? defaultPortAvailable; const portAvailable = options.portAvailable ?? defaultPortAvailable;
if (!(await portAvailable(options.host, options.port))) { if (!(await portAvailable(options.host, options.port))) {

View file

@ -1,16 +1,11 @@
import { randomUUID } from 'node:crypto'; import { randomUUID } from 'node:crypto';
import { createServer, type IncomingHttpHeaders, type IncomingMessage, type Server, type ServerResponse } from 'node:http'; import { createServer, type IncomingHttpHeaders, type IncomingMessage, type Server, type ServerResponse } from 'node:http';
import { createDefaultKtxMcpServer, createLocalProjectMcpContextPorts } from '@ktx/context/mcp'; import { loadKtxProject } from '@ktx/context/project';
import { createLocalProjectMemoryCapture } from '@ktx/context/memory';
import { loadKtxProject, type KtxLocalProject } from '@ktx/context/project';
import type { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'; import type { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js'; import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import { isInitializeRequest } from '@modelcontextprotocol/sdk/types.js'; import { isInitializeRequest } from '@modelcontextprotocol/sdk/types.js';
import type { KtxCliIo } from './cli-runtime.js'; import type { KtxCliIo } from './cli-runtime.js';
import { createKtxCliIngestQueryExecutor } from './ingest-query-executor.js'; import { createKtxMcpServerFactory } from './mcp-server-factory.js';
import { createKtxCliScanConnector } from './local-scan-connectors.js';
import { createManagedPythonSemanticLayerComputePort } from './managed-python-command.js';
import { createManagedDaemonSqlAnalysisPort } from './managed-python-http.js';
const DEFAULT_ALLOWED_HOSTS = ['localhost', '127.0.0.1', '::1'] as const; const DEFAULT_ALLOWED_HOSTS = ['localhost', '127.0.0.1', '::1'] as const;
@ -124,13 +119,6 @@ export interface RunKtxMcpHttpServerOptions extends McpSecurityConfigInput {
loadProject?: typeof loadKtxProject; loadProject?: typeof loadKtxProject;
} }
function noopIo(): KtxCliIo {
return {
stdout: { write() {} },
stderr: { write() {} },
};
}
function writeJson(res: ServerResponse, status: number, body: object): void { function writeJson(res: ServerResponse, status: number, body: object): void {
const payload = `${JSON.stringify(body)}\n`; const payload = `${JSON.stringify(body)}\n`;
res.writeHead(status, { res.writeHead(status, {
@ -159,55 +147,6 @@ async function readJsonBody(req: IncomingMessage): Promise<unknown> {
return raw.trim().length === 0 ? undefined : (JSON.parse(raw) as unknown); return raw.trim().length === 0 ? undefined : (JSON.parse(raw) as unknown);
} }
async function defaultMcpServerFactory(input: {
project: KtxLocalProject;
projectDir: string;
cliVersion: string;
io?: KtxCliIo;
}): Promise<() => McpServer> {
const io = input.io ?? noopIo();
const queryExecutor = createKtxCliIngestQueryExecutor(input.project);
const semanticLayerCompute = await createManagedPythonSemanticLayerComputePort({
cliVersion: input.cliVersion,
installPolicy: 'auto',
io,
});
const sqlAnalysis = createManagedDaemonSqlAnalysisPort({
cliVersion: input.cliVersion,
projectDir: input.projectDir,
installPolicy: 'auto',
io,
});
const contextTools = createLocalProjectMcpContextPorts(input.project, {
semanticLayerCompute,
queryExecutor,
sqlAnalysis,
localScan: {
createConnector: async (connectionId) => createKtxCliScanConnector(input.project, connectionId),
},
localIngest: {
semanticLayerCompute,
queryExecutor,
},
});
let memoryCapture: ReturnType<typeof createLocalProjectMemoryCapture> | undefined;
try {
memoryCapture = createLocalProjectMemoryCapture(input.project, { semanticLayerCompute, queryExecutor });
} catch (error) {
input.io?.stderr.write(`KTX MCP memory_capture disabled: ${error instanceof Error ? error.message : String(error)}\n`);
}
return () =>
createDefaultKtxMcpServer({
name: 'ktx',
version: input.cliVersion,
userContext: { userId: 'local' },
contextTools,
memoryCapture,
});
}
function listenerPort(server: Server, fallback: number): number { function listenerPort(server: Server, fallback: number): number {
const address = server.address(); const address = server.address();
return typeof address === 'object' && address ? address.port : fallback; return typeof address === 'object' && address ? address.port : fallback;
@ -233,7 +172,7 @@ export async function runKtxMcpHttpServer(options: RunKtxMcpHttpServerOptions):
: undefined; : undefined;
const createMcpServer = const createMcpServer =
options.createMcpServer ?? options.createMcpServer ??
(await defaultMcpServerFactory({ (await createKtxMcpServerFactory({
project: project!, project: project!,
projectDir: options.projectDir, projectDir: options.projectDir,
cliVersion: options.cliVersion ?? '0.0.0-private', cliVersion: options.cliVersion ?? '0.0.0-private',

View file

@ -0,0 +1,63 @@
import { createDefaultKtxMcpServer, createLocalProjectMcpContextPorts } from '@ktx/context/mcp';
import { createLocalProjectMemoryIngest } from '@ktx/context/memory';
import type { KtxLocalProject } from '@ktx/context/project';
import type { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import type { KtxCliIo } from './cli-runtime.js';
import { createKtxCliIngestQueryExecutor } from './ingest-query-executor.js';
import { createKtxCliScanConnector } from './local-scan-connectors.js';
import { createManagedPythonSemanticLayerComputePort } from './managed-python-command.js';
import { createManagedDaemonSqlAnalysisPort } from './managed-python-http.js';
function noopMcpIo(): KtxCliIo {
return {
stdout: { write() {} },
stderr: { write() {} },
};
}
export async function createKtxMcpServerFactory(input: {
project: KtxLocalProject;
projectDir: string;
cliVersion: string;
io?: KtxCliIo;
}): Promise<() => McpServer> {
const io = input.io ?? noopMcpIo();
const queryExecutor = createKtxCliIngestQueryExecutor(input.project);
const semanticLayerCompute = await createManagedPythonSemanticLayerComputePort({
cliVersion: input.cliVersion,
installPolicy: 'auto',
io,
});
const sqlAnalysis = createManagedDaemonSqlAnalysisPort({
cliVersion: input.cliVersion,
projectDir: input.projectDir,
installPolicy: 'auto',
io,
});
const contextTools = createLocalProjectMcpContextPorts(input.project, {
semanticLayerCompute,
queryExecutor,
sqlAnalysis,
localScan: {
createConnector: async (connectionId) => createKtxCliScanConnector(input.project, connectionId),
},
});
let memoryIngest: ReturnType<typeof createLocalProjectMemoryIngest> | undefined;
try {
memoryIngest = createLocalProjectMemoryIngest(input.project, { semanticLayerCompute, queryExecutor });
} catch (error) {
input.io?.stderr.write(`KTX MCP memory_ingest disabled: ${error instanceof Error ? error.message : String(error)}\n`);
}
return () =>
createDefaultKtxMcpServer({
name: 'ktx',
version: input.cliVersion,
userContext: { userId: 'local' },
contextTools: {
...contextTools,
...(memoryIngest ? { memoryIngest } : {}),
},
});
}

View file

@ -0,0 +1,64 @@
import process from 'node:process';
import type { Readable, Writable } from 'node:stream';
import { loadKtxProject } from '@ktx/context/project';
import type { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
import type { KtxCliIo } from './cli-runtime.js';
import { createKtxMcpServerFactory } from './mcp-server-factory.js';
export interface RunKtxMcpStdioServerOptions {
projectDir: string;
cliVersion?: string;
io?: KtxCliIo;
createMcpServer?: () => McpServer;
loadProject?: typeof loadKtxProject;
stdin?: Readable;
stdout?: Writable;
}
export async function runKtxMcpStdioServer(options: RunKtxMcpStdioServerOptions): Promise<void> {
const project =
options.createMcpServer === undefined
? await (options.loadProject ?? loadKtxProject)({ projectDir: options.projectDir })
: undefined;
const protocolIo: KtxCliIo = {
stdout: { write() {} },
stderr: options.io?.stderr ?? { write() {} },
};
const createMcpServer =
options.createMcpServer ??
(await createKtxMcpServerFactory({
project: project!,
projectDir: options.projectDir,
cliVersion: options.cliVersion ?? '0.0.0-private',
io: protocolIo,
}));
const stdin = options.stdin ?? process.stdin;
const transport = new StdioServerTransport(stdin, options.stdout);
await new Promise<void>((resolve, reject) => {
let settled = false;
const settle = (callback: () => void) => {
if (settled) return;
settled = true;
stdin.off('end', closeTransport);
stdin.off('close', closeTransport);
callback();
};
const closeTransport = () => {
transport.close().catch((error: unknown) => {
settle(() => reject(error instanceof Error ? error : new Error(String(error))));
});
};
transport.onclose = () => settle(resolve);
transport.onerror = (error) => {
options.io?.stderr.write(`KTX MCP stdio transport error: ${error.message}\n`);
settle(() => reject(error));
};
stdin.once('end', closeTransport);
stdin.once('close', closeTransport);
createMcpServer().connect(transport).catch((error: unknown) => {
settle(() => reject(error instanceof Error ? error : new Error(String(error))));
});
});
}

View file

@ -27,7 +27,7 @@ function replayInput(): MemoryFlowReplayInput {
{ unitKey: 'customers', target: 'sl', action: 'updated', key: 'orbit_demo.customers', summary: 'customer metrics', rawFiles: ['customers'], status: 'success' }, { unitKey: 'customers', target: 'sl', action: 'updated', key: 'orbit_demo.customers', summary: 'customer metrics', rawFiles: ['customers'], status: 'success' },
], ],
provenance: [{ rawPath: 'orders', artifactKind: 'wiki', artifactKey: 'wiki/orders.md', actionType: 'wiki_written' }], provenance: [{ rawPath: 'orders', artifactKind: 'wiki', artifactKey: 'wiki/orders.md', actionType: 'wiki_written' }],
transcripts: [{ unitKey: 'orders', path: '/tmp/t.jsonl', toolCallCount: 2, errorCount: 0, toolNames: ['read_raw_span', 'wiki_write'] }], transcripts: [{ unitKey: 'orders', path: '/tmp/t.jsonl', toolCallCount: 2, errorCount: 0, toolNames: ['read_raw_span', 'memory_ingest'] }],
}, },
events: [ events: [
{ type: 'source_acquired', adapter: 'live-database', trigger: 'manual_resync', fileCount: 2 }, { type: 'source_acquired', adapter: 'live-database', trigger: 'manual_resync', fileCount: 2 },

View file

@ -421,11 +421,18 @@ describe('runKtxPublicIngest', () => {
it('runs query history after schema ingest with current-run window override', async () => { it('runs query history after schema ingest with current-run window override', async () => {
const io = makeIo(); const io = makeIo();
const runtimeIo = makeIo({ isTTY: true });
const project = deepReadyProject({ const project = deepReadyProject({
warehouse: { driver: 'postgres', context: { queryHistory: { enabled: true, windowDays: 90 } } }, warehouse: { driver: 'postgres', context: { queryHistory: { enabled: true, windowDays: 90 } } },
}); });
const runScan = vi.fn(async () => 0); const runScan = vi.fn(async () => 0);
const runIngest = vi.fn<NonNullable<KtxPublicIngestDeps['runIngest']>>(async () => 0); const runIngest = vi.fn<NonNullable<KtxPublicIngestDeps['runIngest']>>(async () => 0);
const deps = {
loadProject: vi.fn(async () => project),
runScan,
runIngest,
runtimeIo: runtimeIo.io,
} as KtxPublicIngestDeps & { runtimeIo: typeof runtimeIo.io };
await expect( await expect(
runKtxPublicIngest( runKtxPublicIngest(
@ -442,13 +449,14 @@ describe('runKtxPublicIngest', () => {
queryHistoryWindowDays: 30, queryHistoryWindowDays: 30,
}, },
io.io, io.io,
{ loadProject: vi.fn(async () => project), runScan, runIngest }, deps,
), ),
).resolves.toBe(0); ).resolves.toBe(0);
expect(runScan).toHaveBeenCalledWith( expect(runScan).toHaveBeenCalledWith(
expect.objectContaining({ connectionId: 'warehouse', mode: 'enriched' }), expect.objectContaining({ connectionId: 'warehouse', mode: 'enriched' }),
expect.anything(), expect.anything(),
expect.objectContaining({ runtimeIo: runtimeIo.io }),
); );
expect(runIngest).toHaveBeenCalledWith( expect(runIngest).toHaveBeenCalledWith(
expect.objectContaining({ expect.objectContaining({
@ -461,6 +469,7 @@ describe('runKtxPublicIngest', () => {
historicSqlPullConfigOverride: expect.objectContaining({ dialect: 'postgres', windowDays: 30 }), historicSqlPullConfigOverride: expect.objectContaining({ dialect: 'postgres', windowDays: 30 }),
}), }),
expect.anything(), expect.anything(),
expect.objectContaining({ runtimeIo: runtimeIo.io }),
); );
}); });

View file

@ -94,6 +94,7 @@ export interface KtxPublicIngestDeps {
) => Promise<{ exitCode: number }>; ) => Promise<{ exitCode: number }>;
scanProgress?: KtxProgressPort; scanProgress?: KtxProgressPort;
ingestProgress?: (update: KtxIngestProgressUpdate) => void; ingestProgress?: (update: KtxIngestProgressUpdate) => void;
runtimeIo?: KtxCliIo;
onPhaseStart?: (phaseKey: KtxPublicIngestPhaseKey) => void; onPhaseStart?: (phaseKey: KtxPublicIngestPhaseKey) => void;
onPhaseEnd?: (phaseKey: KtxPublicIngestPhaseKey, status: 'done' | 'failed' | 'skipped', summary?: string) => void; onPhaseEnd?: (phaseKey: KtxPublicIngestPhaseKey, status: 'done' | 'failed' | 'skipped', summary?: string) => void;
} }
@ -719,10 +720,13 @@ export async function executePublicIngestTarget(
const runScan = deps.runScan ?? runKtxScan; const runScan = deps.runScan ?? runKtxScan;
const capturedScanIo = deps.scanProgress ? null : createCapturedPublicIngestIo(); const capturedScanIo = deps.scanProgress ? null : createCapturedPublicIngestIo();
const scanIo = capturedScanIo ?? io; const scanIo = capturedScanIo ?? io;
const scanDeps = {
...(deps.scanProgress ? { progress: deps.scanProgress } : {}),
...(deps.runtimeIo ? { runtimeIo: deps.runtimeIo } : {}),
};
deps.onPhaseStart?.('database-schema'); deps.onPhaseStart?.('database-schema');
const scanExitCode = deps.scanProgress const scanExitCode =
? await runScan(scanArgs, scanIo, { progress: deps.scanProgress }) Object.keys(scanDeps).length > 0 ? await runScan(scanArgs, scanIo, scanDeps) : await runScan(scanArgs, scanIo);
: await runScan(scanArgs, scanIo);
if (scanExitCode !== 0) { if (scanExitCode !== 0) {
deps.onPhaseEnd?.('database-schema', 'failed'); deps.onPhaseEnd?.('database-schema', 'failed');
if (target.queryHistory?.enabled === true) { if (target.queryHistory?.enabled === true) {
@ -759,10 +763,15 @@ export async function executePublicIngestTarget(
}; };
const capturedIngestIo = deps.ingestProgress ? null : createCapturedPublicIngestIo(); const capturedIngestIo = deps.ingestProgress ? null : createCapturedPublicIngestIo();
const ingestIo = capturedIngestIo ?? io; const ingestIo = capturedIngestIo ?? io;
const ingestDeps = {
...(deps.ingestProgress ? { progress: deps.ingestProgress } : {}),
...(deps.runtimeIo ? { runtimeIo: deps.runtimeIo } : {}),
};
deps.onPhaseStart?.('query-history'); deps.onPhaseStart?.('query-history');
const qhExitCode = deps.ingestProgress const qhExitCode =
? await runIngest(ingestArgs, ingestIo, { progress: deps.ingestProgress }) Object.keys(ingestDeps).length > 0
: await runIngest(ingestArgs, ingestIo); ? await runIngest(ingestArgs, ingestIo, ingestDeps)
: await runIngest(ingestArgs, ingestIo);
if (qhExitCode !== 0) { if (qhExitCode !== 0) {
deps.onPhaseEnd?.('query-history', 'failed'); deps.onPhaseEnd?.('query-history', 'failed');
return markTargetResult( return markTargetResult(
@ -795,10 +804,15 @@ export async function executePublicIngestTarget(
const runIngest = deps.runIngest ?? runKtxIngest; const runIngest = deps.runIngest ?? runKtxIngest;
const capturedIngestIo = deps.ingestProgress ? null : createCapturedPublicIngestIo(); const capturedIngestIo = deps.ingestProgress ? null : createCapturedPublicIngestIo();
const ingestIo = capturedIngestIo ?? io; const ingestIo = capturedIngestIo ?? io;
const ingestDeps = {
...(deps.ingestProgress ? { progress: deps.ingestProgress } : {}),
...(deps.runtimeIo ? { runtimeIo: deps.runtimeIo } : {}),
};
deps.onPhaseStart?.('source-ingest'); deps.onPhaseStart?.('source-ingest');
const exitCode = deps.ingestProgress const exitCode =
? await runIngest(ingestArgs, ingestIo, { progress: deps.ingestProgress }) Object.keys(ingestDeps).length > 0
: await runIngest(ingestArgs, ingestIo); ? await runIngest(ingestArgs, ingestIo, ingestDeps)
: await runIngest(ingestArgs, ingestIo);
deps.onPhaseEnd?.('source-ingest', exitCode === 0 ? 'done' : 'failed'); deps.onPhaseEnd?.('source-ingest', exitCode === 0 ? 'done' : 'failed');
return markTargetResult( return markTargetResult(
target, target,

View file

@ -9,7 +9,7 @@ import type {
RunLocalScanOptions, RunLocalScanOptions,
} from '@ktx/context/scan'; } from '@ktx/context/scan';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { createCliScanProgress, runKtxScan } from './scan.js'; import { createCliScanProgress, runKtxScan, type KtxScanDeps } from './scan.js';
const sqlServerExtractSchema = vi.hoisted(() => const sqlServerExtractSchema = vi.hoisted(() =>
vi.fn(async (connectionId: string) => ({ vi.fn(async (connectionId: string) => ({
@ -392,6 +392,7 @@ describe('runKtxScan', () => {
}), }),
); );
const io = makeIo(); const io = makeIo();
const runtimeIo = makeIo({ isTTY: true });
await expect( await expect(
runKtxScan( runKtxScan(
@ -406,7 +407,9 @@ describe('runKtxScan', () => {
runtimeInstallPolicy: 'auto', runtimeInstallPolicy: 'auto',
}, },
io.io, io.io,
{ runLocalScan, createLocalIngestAdapters }, { runLocalScan, createLocalIngestAdapters, runtimeIo: runtimeIo.io } as KtxScanDeps & {
runtimeIo: typeof runtimeIo.io;
},
), ),
).resolves.toBe(0); ).resolves.toBe(0);
@ -415,7 +418,7 @@ describe('runKtxScan', () => {
cliVersion: '0.2.0', cliVersion: '0.2.0',
projectDir: tempDir, projectDir: tempDir,
installPolicy: 'auto', installPolicy: 'auto',
io: io.io, io: runtimeIo.io,
}, },
}); });
}); });

View file

@ -30,6 +30,7 @@ export interface KtxScanDeps {
runLocalScan?: typeof runLocalScan; runLocalScan?: typeof runLocalScan;
createLocalIngestAdapters?: typeof createKtxCliLocalIngestAdapters; createLocalIngestAdapters?: typeof createKtxCliLocalIngestAdapters;
progress?: KtxProgressPort; progress?: KtxProgressPort;
runtimeIo?: KtxCliIo;
} }
function shouldUseStyledOutput(io: KtxCliIo): boolean { function shouldUseStyledOutput(io: KtxCliIo): boolean {
@ -313,7 +314,7 @@ export function createCliScanProgress(
export async function runKtxScan(args: KtxScanArgs, io: KtxCliIo = process, deps: KtxScanDeps = {}): Promise<number> { export async function runKtxScan(args: KtxScanArgs, io: KtxCliIo = process, deps: KtxScanDeps = {}): Promise<number> {
try { try {
const project = await loadKtxProject({ projectDir: args.projectDir }); const project = await loadKtxProject({ projectDir: args.projectDir });
const managedDaemon = managedDaemonOptionsForScanRun(args, io); const managedDaemon = managedDaemonOptionsForScanRun(args, deps.runtimeIo ?? io);
const connector = const connector =
args.mode !== 'structural' || args.detectRelationships args.mode !== 'structural' || args.detectRelationships
? await createKtxCliScanConnector(project, args.connectionId) ? await createKtxCliScanConnector(project, args.connectionId)

View file

@ -2,6 +2,7 @@ import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'; import { tmpdir } from 'node:os';
import { join } from 'node:path'; import { join } from 'node:path';
import { readKtxSetupState } from '@ktx/context/project'; import { readKtxSetupState } from '@ktx/context/project';
import { strFromU8, unzipSync } from 'fflate';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { import {
formatInstallSummary, formatInstallSummary,
@ -24,6 +25,44 @@ function makeIo() {
}; };
} }
async function readZipText(path: string, entry: string): Promise<string> {
const archive = unzipSync(new Uint8Array(await readFile(path)));
const content = archive[entry];
if (!content) throw new Error(`Missing zip entry: ${entry}`);
return strFromU8(content);
}
function captureEnvKeys(env: NodeJS.ProcessEnv, keys: readonly string[]): Record<string, string | undefined> {
const snapshot: Record<string, string | undefined> = {};
for (const key of keys) snapshot[key] = env[key];
return snapshot;
}
function clearEnvKeys(env: NodeJS.ProcessEnv, keys: readonly string[]): void {
for (const key of keys) delete env[key];
}
function captureKtxEnv(env: NodeJS.ProcessEnv): Record<string, string | undefined> {
const snapshot: Record<string, string | undefined> = {};
for (const key of Object.keys(env)) {
if (key.startsWith('KTX_')) snapshot[key] = env[key];
}
return snapshot;
}
function clearKtxEnv(env: NodeJS.ProcessEnv): void {
for (const key of Object.keys(env)) {
if (key.startsWith('KTX_')) delete env[key];
}
}
function restoreEnvKeys(env: NodeJS.ProcessEnv, snapshot: Record<string, string | undefined>): void {
for (const [key, value] of Object.entries(snapshot)) {
if (value === undefined) delete env[key];
else env[key] = value;
}
}
describe('setup agents', () => { describe('setup agents', () => {
let tempDir: string; let tempDir: string;
@ -37,28 +76,54 @@ describe('setup agents', () => {
await rm(tempDir, { recursive: true, force: true }); await rm(tempDir, { recursive: true, force: true });
}); });
it('plans project-scoped CLI and research files for every target', () => { it('plans project-scoped MCP analytics files for every target', () => {
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'claude-code', scope: 'project', mode: 'cli' })).toEqual([ expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'claude-code', scope: 'project', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'claude-desktop', scope: 'global', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.ktx/agents/claude/ktx-plugin-runner.sh'), role: 'launcher' },
{ kind: 'file', path: join(tempDir, '.ktx/agents/claude/ktx-plugin.zip'), role: 'claude-plugin' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'codex', scope: 'project', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'cursor', scope: 'project', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx-analytics.mdc'), role: 'analytics-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'opencode', scope: 'project', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.opencode/commands/ktx-analytics.md'), role: 'analytics-skill' },
]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'universal', scope: 'project', mode: 'mcp' })).toEqual([
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
]);
});
it('plans project-scoped admin CLI files for every target when requested', () => {
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'claude-code', scope: 'project', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx/SKILL.md'), role: 'skill' }, { kind: 'file', path: join(tempDir, '.claude/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx-research/SKILL.md'), role: 'research-skill' },
{ kind: 'file', path: join(tempDir, '.claude/rules/ktx.md'), role: 'rule' }, { kind: 'file', path: join(tempDir, '.claude/rules/ktx.md'), role: 'rule' },
]); ]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'codex', scope: 'project', mode: 'cli' })).toEqual([ expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'codex', scope: 'project', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx/SKILL.md'), role: 'skill' }, { kind: 'file', path: join(tempDir, '.agents/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-research/SKILL.md'), role: 'research-skill' },
{ kind: 'file', path: join(tempDir, '.codex/instructions/ktx.md'), role: 'rule' }, { kind: 'file', path: join(tempDir, '.codex/instructions/ktx.md'), role: 'rule' },
]); ]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'cursor', scope: 'project', mode: 'cli' })).toEqual([ expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'cursor', scope: 'project', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx-analytics.mdc'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx.mdc') }, { kind: 'file', path: join(tempDir, '.cursor/rules/ktx.mdc') },
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx-research.mdc'), role: 'research-skill' },
]); ]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'opencode', scope: 'project', mode: 'cli' })).toEqual([ expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'opencode', scope: 'project', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.opencode/commands/ktx-analytics.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.opencode/commands/ktx.md') }, { kind: 'file', path: join(tempDir, '.opencode/commands/ktx.md') },
{ kind: 'file', path: join(tempDir, '.opencode/commands/ktx-research.md'), role: 'research-skill' },
]); ]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'universal', scope: 'project', mode: 'cli' })).toEqual([ expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'universal', scope: 'project', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx/SKILL.md') }, { kind: 'file', path: join(tempDir, '.agents/skills/ktx/SKILL.md') },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-research/SKILL.md'), role: 'research-skill' }, ]);
expect(plannedKtxAgentFiles({ projectDir: tempDir, target: 'claude-desktop', scope: 'global', mode: 'mcp-cli' })).toEqual([
{ kind: 'file', path: join(tempDir, '.ktx/agents/claude/ktx-plugin-runner.sh'), role: 'launcher' },
{ kind: 'file', path: join(tempDir, '.ktx/agents/claude/ktx-plugin.zip'), role: 'claude-plugin' },
]); ]);
}); });
@ -74,7 +139,7 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'universal', target: 'universal',
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
@ -82,7 +147,7 @@ describe('setup agents', () => {
).resolves.toEqual({ ).resolves.toEqual({
status: 'ready', status: 'ready',
projectDir: tempDir, projectDir: tempDir,
installs: [{ target: 'universal', scope: 'project', mode: 'cli' }], installs: [{ target: 'universal', scope: 'project', mode: 'mcp-cli' }],
}); });
await expect(stat(join(tempDir, '.agents/skills/ktx/SKILL.md'))).resolves.toBeDefined(); await expect(stat(join(tempDir, '.agents/skills/ktx/SKILL.md'))).resolves.toBeDefined();
@ -99,13 +164,13 @@ describe('setup agents', () => {
expect(await readKtxAgentInstallManifest(tempDir)).toMatchObject({ expect(await readKtxAgentInstallManifest(tempDir)).toMatchObject({
version: 1, version: 1,
projectDir: tempDir, projectDir: tempDir,
installs: [{ target: 'universal', scope: 'project', mode: 'cli' }], installs: [{ target: 'universal', scope: 'project', mode: 'mcp-cli' }],
}); });
expect(await readKtxSetupState(tempDir)).toEqual({ completed_steps: ['agents'] }); expect(await readKtxSetupState(tempDir)).toEqual({ completed_steps: ['agents'] });
expect(io.stderr()).toBe(''); expect(io.stderr()).toBe('');
}); });
it('installs the research skill from the runtime asset', async () => { it('installs the analytics skill from the runtime asset', async () => {
const io = makeIo(); const io = makeIo();
await expect( await expect(
@ -117,17 +182,20 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'universal', target: 'universal',
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
), ),
).resolves.toMatchObject({ status: 'ready' }); ).resolves.toMatchObject({ status: 'ready' });
const researchSkill = await readFile(join(tempDir, '.agents/skills/ktx-research/SKILL.md'), 'utf-8'); const analyticsSkill = await readFile(join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), 'utf-8');
expect(researchSkill).toContain('name: ktx-research'); expect(analyticsSkill).toContain('name: ktx-analytics');
expect(researchSkill).toContain('Always run `discover_data` before writing SQL.'); expect(analyticsSkill).toContain('Always run `discover_data` before writing SQL.');
expect(researchSkill).toContain('Treat a `dictionary_search` miss as non-authoritative.'); expect(analyticsSkill).toContain('Treat a `dictionary_search` miss as non-authoritative.');
expect(analyticsSkill).toContain('memory_ingest');
expect(analyticsSkill).toContain('ARR is reported in cents');
expect(analyticsSkill).not.toContain(`memory_${'capture'}`);
}); });
it('writes PATH-independent launcher commands for skills', async () => { it('writes PATH-independent launcher commands for skills', async () => {
@ -142,7 +210,7 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'universal', target: 'universal',
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
@ -170,7 +238,7 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'claude-code', target: 'claude-code',
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
@ -187,6 +255,279 @@ describe('setup agents', () => {
expect(io.stdout()).toContain('Run `ktx mcp start` to enable the configured KTX MCP server.'); expect(io.stdout()).toContain('Run `ktx mcp start` to enable the configured KTX MCP server.');
}); });
it('prompts for MCP-first client agent connection mode in interactive setup', async () => {
const io = makeIo();
const prompts = {
select: vi.fn(async ({ message }: { message: string }) => (message.startsWith('Where') ? 'project' : 'mcp')),
multiselect: vi.fn(async () => ['claude-code']),
cancel: vi.fn(),
};
await expect(
runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'auto',
yes: false,
agents: true,
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
io.io,
{ prompts },
),
).resolves.toMatchObject({
status: 'ready',
installs: [{ target: 'claude-code', scope: 'project', mode: 'mcp' }],
});
expect(prompts.select).toHaveBeenCalledWith({
message: 'How should client agents connect to this KTX project?',
options: [
{ value: 'mcp', label: 'MCP tools + analytics skill' },
{ value: 'mcp-cli', label: 'MCP tools + analytics skill + admin CLI skill' },
],
});
expect(prompts.multiselect).toHaveBeenCalledWith(
expect.objectContaining({
options: expect.arrayContaining([{ value: 'claude-desktop', label: 'Claude Desktop' }]),
}),
);
});
it('prompts for global scope when every selected target supports it', async () => {
const home = await mkdtemp(join(tmpdir(), 'ktx-setup-agents-home-'));
const previousHome = process.env.HOME;
process.env.HOME = home;
try {
const io = makeIo();
const prompts = {
select: vi.fn(async ({ message }: { message: string }) =>
message.startsWith('Where should') ? 'global' : 'mcp',
),
multiselect: vi.fn(async () => ['claude-code']),
cancel: vi.fn(),
};
await expect(
runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'auto',
yes: false,
agents: true,
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
io.io,
{ prompts },
),
).resolves.toMatchObject({
status: 'ready',
installs: [{ target: 'claude-code', scope: 'global', mode: 'mcp' }],
});
expect(prompts.select).toHaveBeenCalledWith({
message: 'Where should KTX install supported agent config?',
options: [
{ value: 'project', label: 'Project' },
{ value: 'global', label: 'Global' },
],
});
} finally {
process.env.HOME = previousHome;
await rm(home, { recursive: true, force: true });
}
});
it('registers Claude Desktop MCP via claude_desktop_config.json and ships a skills-only plugin', async () => {
const home = await mkdtemp(join(tmpdir(), 'ktx-setup-agents-home-'));
const previousHome = process.env.HOME;
const envSnapshot = captureEnvKeys(process.env, ['OPENAI_API_KEY', 'ANTHROPIC_API_KEY']);
const ktxEnvSnapshot = captureKtxEnv(process.env);
process.env.HOME = home;
clearEnvKeys(process.env, ['OPENAI_API_KEY', 'ANTHROPIC_API_KEY']);
clearKtxEnv(process.env);
try {
const io = makeIo();
await expect(
runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'claude-desktop',
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
io.io,
),
).resolves.toMatchObject({
status: 'ready',
installs: [{ target: 'claude-desktop', scope: 'global', mode: 'mcp' }],
});
const pluginPath = join(tempDir, '.ktx/agents/claude/ktx-plugin.zip');
const launcherPath = join(tempDir, '.ktx/agents/claude/ktx-plugin-runner.sh');
await expect(stat(pluginPath)).resolves.toBeDefined();
const launcherStat = await stat(launcherPath);
expect(launcherStat.mode & 0o111).not.toBe(0);
const launcher = await readFile(launcherPath, 'utf-8');
expect(launcher).toContain('KTX_CLI_BIN=');
expect(launcher).toContain('.nvm/versions/node');
const configPath = join(home, 'Library/Application Support/Claude/claude_desktop_config.json');
const config = JSON.parse(await readFile(configPath, 'utf-8')) as {
mcpServers: { ktx: { command: string; args: string[]; env?: Record<string, string> } };
};
expect(config.mcpServers.ktx).toEqual({
command: launcherPath,
args: ['--project-dir', tempDir, 'mcp', 'stdio'],
});
expect(await readZipText(pluginPath, '.claude-plugin/plugin.json')).toContain('"name": "ktx"');
await expect(readZipText(pluginPath, '.mcp.json')).rejects.toThrow('Missing zip entry');
expect(await readZipText(pluginPath, 'skills/ktx-analytics/SKILL.md')).toContain('KTX Analytics Workflow');
const setupMd = await readZipText(pluginPath, 'SETUP.md');
expect(setupMd).not.toContain('ktx mcp start');
expect(setupMd).toContain('claude_desktop_config.json');
await expect(readZipText(pluginPath, 'skills/ktx/SKILL.md')).rejects.toThrow('Missing zip entry');
expect(io.stdout()).toContain('Claude plugin generated');
expect(io.stdout()).toContain('.ktx/agents/claude/ktx-plugin.zip');
expect(io.stdout()).toContain('KTX MCP server registered');
expect(io.stdout()).toContain('claude_desktop_config.json');
expect(io.stdout()).toContain('Restart Claude Desktop');
expect(io.stdout()).not.toContain('Run `ktx mcp start`');
} finally {
process.env.HOME = previousHome;
restoreEnvKeys(process.env, envSnapshot);
restoreEnvKeys(process.env, ktxEnvSnapshot);
await rm(home, { recursive: true, force: true });
}
});
it('captures KTX_*, OPENAI_API_KEY, and ANTHROPIC_API_KEY into the Claude Desktop MCP env block', async () => {
const home = await mkdtemp(join(tmpdir(), 'ktx-setup-agents-home-'));
const previousHome = process.env.HOME;
const envSnapshot = captureEnvKeys(process.env, [
'OPENAI_API_KEY',
'ANTHROPIC_API_KEY',
'KTX_LOG_LEVEL',
]);
const ktxEnvSnapshot = captureKtxEnv(process.env);
process.env.HOME = home;
clearKtxEnv(process.env);
process.env.OPENAI_API_KEY = 'sk-test-openai'; // pragma: allowlist secret
process.env.ANTHROPIC_API_KEY = 'sk-ant-test'; // pragma: allowlist secret
process.env.KTX_LOG_LEVEL = 'debug';
try {
const io = makeIo();
await runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'claude-desktop',
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
io.io,
);
const configPath = join(home, 'Library/Application Support/Claude/claude_desktop_config.json');
const config = JSON.parse(await readFile(configPath, 'utf-8')) as {
mcpServers: { ktx: { env?: Record<string, string> } };
};
expect(config.mcpServers.ktx.env).toEqual({
OPENAI_API_KEY: 'sk-test-openai', // pragma: allowlist secret
ANTHROPIC_API_KEY: 'sk-ant-test', // pragma: allowlist secret
KTX_LOG_LEVEL: 'debug',
});
} finally {
process.env.HOME = previousHome;
restoreEnvKeys(process.env, envSnapshot);
restoreEnvKeys(process.env, ktxEnvSnapshot);
await rm(home, { recursive: true, force: true });
}
});
it('includes the admin CLI skill in the Claude Desktop plugin zip when requested', async () => {
const home = await mkdtemp(join(tmpdir(), 'ktx-setup-agents-home-'));
const previousHome = process.env.HOME;
process.env.HOME = home;
try {
const io = makeIo();
await expect(
runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'claude-desktop',
scope: 'project',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
),
).resolves.toMatchObject({
status: 'ready',
installs: [{ target: 'claude-desktop', scope: 'global', mode: 'mcp-cli' }],
});
const pluginPath = join(tempDir, '.ktx/agents/claude/ktx-plugin.zip');
const adminSkill = await readZipText(pluginPath, 'skills/ktx/SKILL.md');
expect(adminSkill).toContain(`--project-dir ${tempDir}`);
expect(adminSkill).toContain('status --json');
expect(await readZipText(pluginPath, 'SETUP.md')).toContain('admin CLI skill');
await expect(readZipText(pluginPath, '.mcp.json')).rejects.toThrow('Missing zip entry');
} finally {
process.env.HOME = previousHome;
await rm(home, { recursive: true, force: true });
}
});
it('installs MCP client config and analytics skill without admin CLI files', async () => {
const io = makeIo();
await expect(
runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'claude-code',
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
io.io,
),
).resolves.toMatchObject({
status: 'ready',
installs: [{ target: 'claude-code', scope: 'project', mode: 'mcp' }],
});
const mcpJson = JSON.parse(await readFile(join(tempDir, '.mcp.json'), 'utf-8')) as {
mcpServers: { ktx: { type: string; url: string } };
};
expect(mcpJson.mcpServers.ktx).toEqual({ type: 'http', url: 'http://localhost:7878/mcp' });
await expect(stat(join(tempDir, '.claude/skills/ktx-analytics/SKILL.md'))).resolves.toBeDefined();
await expect(stat(join(tempDir, '.claude/skills/ktx/SKILL.md'))).rejects.toThrow();
await expect(stat(join(tempDir, '.claude/rules/ktx.md'))).rejects.toThrow();
});
it('writes Cursor project MCP config', async () => { it('writes Cursor project MCP config', async () => {
const io = makeIo(); const io = makeIo();
@ -198,7 +539,7 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'cursor', target: 'cursor',
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
@ -210,7 +551,7 @@ describe('setup agents', () => {
expect(cursorJson.mcpServers.ktx).toEqual({ url: 'http://localhost:7878/mcp' }); expect(cursorJson.mcpServers.ktx).toEqual({ url: 'http://localhost:7878/mcp' });
}); });
it('prints Codex and opencode snippets without mutating printed-only config files', async () => { it('prints Codex, opencode, and universal snippets without mutating printed-only config files', async () => {
const codexIo = makeIo(); const codexIo = makeIo();
await runKtxSetupAgentsStep( await runKtxSetupAgentsStep(
{ {
@ -220,7 +561,7 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'codex', target: 'codex',
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
codexIo.io, codexIo.io,
@ -237,7 +578,7 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'opencode', target: 'opencode',
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
opencodeIo.io, opencodeIo.io,
@ -245,6 +586,23 @@ describe('setup agents', () => {
expect(opencodeIo.stdout()).toContain('"mcp"'); expect(opencodeIo.stdout()).toContain('"mcp"');
expect(opencodeIo.stdout()).toContain('"type": "remote"'); expect(opencodeIo.stdout()).toContain('"type": "remote"');
await expect(readFile(join(tempDir, 'opencode.json'), 'utf-8')).rejects.toThrow(); await expect(readFile(join(tempDir, 'opencode.json'), 'utf-8')).rejects.toThrow();
const universalIo = makeIo();
await runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'universal',
scope: 'project',
mode: 'mcp',
skipAgents: false,
},
universalIo.io,
);
expect(universalIo.stdout()).toContain('Universal MCP endpoint:');
expect(universalIo.stdout()).toContain('http://localhost:7878/mcp');
}); });
it('uses MCP daemon state for port and token metadata without rendering literal tokens', async () => { it('uses MCP daemon state for port and token metadata without rendering literal tokens', async () => {
@ -280,7 +638,7 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'claude-code', target: 'claude-code',
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
@ -314,7 +672,7 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'claude-code', target: 'claude-code',
scope: 'local', scope: 'local',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
@ -340,7 +698,7 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'claude-code', target: 'claude-code',
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
@ -355,6 +713,50 @@ describe('setup agents', () => {
await expect(readKtxAgentInstallManifest(tempDir)).resolves.toEqual(null); await expect(readKtxAgentInstallManifest(tempDir)).resolves.toEqual(null);
}); });
it('removes generated Claude Desktop plugin from the manifest', async () => {
const home = await mkdtemp(join(tmpdir(), 'ktx-setup-agents-home-'));
const previousHome = process.env.HOME;
process.env.HOME = home;
try {
const io = makeIo();
await runKtxSetupAgentsStep(
{
projectDir: tempDir,
inputMode: 'disabled',
yes: true,
agents: true,
target: 'claude-desktop',
scope: 'project',
mode: 'mcp-cli',
skipAgents: false,
},
io.io,
);
const pluginPath = join(tempDir, '.ktx/agents/claude/ktx-plugin.zip');
const launcherPath = join(tempDir, '.ktx/agents/claude/ktx-plugin-runner.sh');
const configPath = join(home, 'Library/Application Support/Claude/claude_desktop_config.json');
await expect(stat(pluginPath)).resolves.toBeDefined();
await expect(stat(launcherPath)).resolves.toBeDefined();
const beforeConfig = JSON.parse(await readFile(configPath, 'utf-8')) as {
mcpServers: Record<string, unknown>;
};
expect(beforeConfig.mcpServers.ktx).toBeDefined();
await expect(removeKtxAgentInstall(tempDir, io.io)).resolves.toBe(0);
await expect(stat(pluginPath)).rejects.toThrow();
await expect(stat(launcherPath)).rejects.toThrow();
const afterConfig = JSON.parse(await readFile(configPath, 'utf-8')) as {
mcpServers: Record<string, unknown>;
};
expect(afterConfig.mcpServers.ktx).toBeUndefined();
await expect(readKtxAgentInstallManifest(tempDir)).resolves.toEqual(null);
} finally {
process.env.HOME = previousHome;
await rm(home, { recursive: true, force: true });
}
});
it('treats cancel as skip in interactive mode', async () => { it('treats cancel as skip in interactive mode', async () => {
const io = makeIo(); const io = makeIo();
const prompts = { const prompts = {
@ -371,7 +773,7 @@ describe('setup agents', () => {
yes: false, yes: false,
agents: true, agents: true,
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
@ -383,7 +785,7 @@ describe('setup agents', () => {
it('explains how to select multiple agent targets in interactive mode', async () => { it('explains how to select multiple agent targets in interactive mode', async () => {
const io = makeIo(); const io = makeIo();
const prompts = { const prompts = {
select: vi.fn(async () => 'cli'), select: vi.fn(async () => 'mcp-cli'),
multiselect: vi.fn(async () => ['back']), multiselect: vi.fn(async () => ['back']),
cancel: vi.fn(), cancel: vi.fn(),
}; };
@ -396,7 +798,7 @@ describe('setup agents', () => {
yes: false, yes: false,
agents: true, agents: true,
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
@ -423,7 +825,7 @@ describe('setup agents', () => {
agents: true, agents: true,
target: 'claude-code', target: 'claude-code',
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io.io, io.io,
@ -432,21 +834,28 @@ describe('setup agents', () => {
const output = io.stdout(); const output = io.stdout();
expect(output).toContain('Agent integration complete'); expect(output).toContain('Agent integration complete');
expect(output).toContain('Claude Code'); expect(output).toContain('Claude Code');
expect(output).toContain('+ Skill installed — teaches your agent which KTX commands to run'); expect(output).toContain('+ Analytics skill installed — teaches your agent the KTX MCP analytics workflow');
expect(output).toContain('.claude/skills/ktx-analytics/SKILL.md');
expect(output).toContain('+ Skill installed — teaches admin agents which KTX CLI commands to run');
expect(output).toContain('.claude/skills/ktx/SKILL.md'); expect(output).toContain('.claude/skills/ktx/SKILL.md');
expect(output).toContain('+ Rule installed — tells your agent when to use KTX'); expect(output).toContain('+ Rule installed — tells admin agents when to use KTX CLI');
expect(output).toContain('.claude/rules/ktx.md'); expect(output).toContain('.claude/rules/ktx.md');
}); });
it('formats summary with relative paths for project scope', () => { it('formats summary with relative paths for project scope', () => {
const summary = formatInstallSummary( const summary = formatInstallSummary(
[{ target: 'cursor', scope: 'project', mode: 'cli' }], [{ target: 'cursor', scope: 'project', mode: 'mcp-cli' }],
[{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx.mdc') }], [
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx-analytics.mdc'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.cursor/rules/ktx.mdc') },
],
tempDir, tempDir,
); );
expect(summary).toContain('Cursor'); expect(summary).toContain('Cursor');
expect(summary).toContain('+ Rule installed — tells your agent when to use KTX'); expect(summary).toContain('+ Analytics skill installed — teaches your agent the KTX MCP analytics workflow');
expect(summary).toContain('.cursor/rules/ktx-analytics.mdc');
expect(summary).toContain('+ Rule installed — tells admin agents when to use KTX CLI');
expect(summary).toContain('.cursor/rules/ktx.mdc'); expect(summary).toContain('.cursor/rules/ktx.mdc');
expect(summary).not.toContain(tempDir); expect(summary).not.toContain(tempDir);
}); });
@ -454,12 +863,14 @@ describe('setup agents', () => {
it('formats summary with multiple agent targets', () => { it('formats summary with multiple agent targets', () => {
const summary = formatInstallSummary( const summary = formatInstallSummary(
[ [
{ target: 'claude-code', scope: 'project', mode: 'cli' }, { target: 'claude-code', scope: 'project', mode: 'mcp-cli' },
{ target: 'codex', scope: 'project', mode: 'cli' }, { target: 'codex', scope: 'project', mode: 'mcp-cli' },
], ],
[ [
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.claude/skills/ktx/SKILL.md'), role: 'skill' }, { kind: 'file', path: join(tempDir, '.claude/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(tempDir, '.claude/rules/ktx.md'), role: 'rule' }, { kind: 'file', path: join(tempDir, '.claude/rules/ktx.md'), role: 'rule' },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
{ kind: 'file', path: join(tempDir, '.agents/skills/ktx/SKILL.md'), role: 'skill' }, { kind: 'file', path: join(tempDir, '.agents/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(tempDir, '.codex/instructions/ktx.md'), role: 'rule' }, { kind: 'file', path: join(tempDir, '.codex/instructions/ktx.md'), role: 'rule' },
], ],
@ -467,9 +878,11 @@ describe('setup agents', () => {
); );
expect(summary).toContain('Claude Code'); expect(summary).toContain('Claude Code');
expect(summary).toContain('+ Skill installed — teaches your agent which KTX commands to run'); expect(summary).toContain('+ Analytics skill installed — teaches your agent the KTX MCP analytics workflow');
expect(summary).toContain('+ Rule installed — tells your agent when to use KTX'); expect(summary).toContain('+ Skill installed — teaches admin agents which KTX CLI commands to run');
expect(summary).toContain('+ Rule installed — tells admin agents when to use KTX CLI');
expect(summary).toContain('Codex'); expect(summary).toContain('Codex');
expect(summary).toContain('.agents/skills/ktx-analytics/SKILL.md');
expect(summary).toContain('.agents/skills/ktx/SKILL.md'); expect(summary).toContain('.agents/skills/ktx/SKILL.md');
}); });
}); });

View file

@ -1,5 +1,5 @@
import { existsSync } from 'node:fs'; import { existsSync } from 'node:fs';
import { mkdir, readFile, rm, writeFile } from 'node:fs/promises'; import { chmod, mkdir, readFile, rm, writeFile } from 'node:fs/promises';
import { dirname, join, relative, resolve } from 'node:path'; import { dirname, join, relative, resolve } from 'node:path';
import { fileURLToPath } from 'node:url'; import { fileURLToPath } from 'node:url';
import { import {
@ -7,17 +7,20 @@ import {
markKtxSetupStateStepComplete, markKtxSetupStateStepComplete,
serializeKtxProjectConfig, serializeKtxProjectConfig,
} from '@ktx/context/project'; } from '@ktx/context/project';
import { strToU8, zipSync } from 'fflate';
import type { KtxCliIo } from './cli-runtime.js'; import type { KtxCliIo } from './cli-runtime.js';
import { bold, dim, green } from './io/symbols.js';
import { withMultiselectNavigation } from './prompt-navigation.js'; import { withMultiselectNavigation } from './prompt-navigation.js';
import { import {
createKtxSetupPromptAdapter, createKtxSetupPromptAdapter,
createKtxSetupUiAdapter,
type KtxSetupPromptOption, type KtxSetupPromptOption,
} from './setup-prompts.js'; } from './setup-prompts.js';
import { readKtxMcpDaemonStatus } from './managed-mcp-daemon.js'; import { readKtxMcpDaemonStatus } from './managed-mcp-daemon.js';
export type KtxAgentTarget = 'claude-code' | 'codex' | 'cursor' | 'opencode' | 'universal'; export type KtxAgentTarget = 'claude-code' | 'claude-desktop' | 'codex' | 'cursor' | 'opencode' | 'universal';
export type KtxAgentScope = 'project' | 'global' | 'local'; export type KtxAgentScope = 'project' | 'global' | 'local';
export type KtxAgentInstallMode = 'cli'; export type KtxAgentInstallMode = 'mcp' | 'mcp-cli';
export interface KtxSetupAgentsArgs { export interface KtxSetupAgentsArgs {
projectDir: string; projectDir: string;
@ -47,7 +50,7 @@ export interface KtxAgentInstallManifest {
installedAt: string; installedAt: string;
installs: Array<{ target: KtxAgentTarget; scope: KtxAgentScope; mode: KtxAgentInstallMode }>; installs: Array<{ target: KtxAgentTarget; scope: KtxAgentScope; mode: KtxAgentInstallMode }>;
entries: Array< entries: Array<
| { kind: 'file'; path: string; role?: 'skill' | 'rule' | 'research-skill' } | { kind: 'file'; path: string; role?: 'skill' | 'rule' | 'analytics-skill' | 'claude-plugin' | 'launcher' }
| { kind: 'json-key'; path: string; jsonPath: string[] } | { kind: 'json-key'; path: string; jsonPath: string[] }
>; >;
} }
@ -169,6 +172,14 @@ function opencodeSnippet(endpoint: KtxMcpEndpointInfo): string {
); );
} }
function universalMcpSnippet(endpoint: KtxMcpEndpointInfo): string {
return [
'Universal MCP endpoint:',
endpoint.url,
...(endpoint.tokenAuth ? ['Header: Authorization: Bearer ${KTX_MCP_TOKEN}'] : []),
].join('\n');
}
function claudeConfigPath(projectDir: string, scope: KtxAgentScope): { path: string; jsonPath: string[] } { function claudeConfigPath(projectDir: string, scope: KtxAgentScope): { path: string; jsonPath: string[] } {
const home = process.env.HOME ?? ''; const home = process.env.HOME ?? '';
if (scope === 'global') { if (scope === 'global') {
@ -188,16 +199,63 @@ function cursorConfigPath(projectDir: string, scope: KtxAgentScope): { path: str
}; };
} }
function claudeDesktopConfigPath(): { path: string; jsonPath: string[] } {
const home = process.env.HOME ?? '';
const path =
process.platform === 'win32'
? join(process.env.APPDATA ?? join(home, 'AppData/Roaming'), 'Claude/claude_desktop_config.json')
: join(home, 'Library/Application Support/Claude/claude_desktop_config.json');
return { path, jsonPath: ['mcpServers', 'ktx'] };
}
const CLAUDE_DESKTOP_FORWARDED_ENV_KEYS = ['OPENAI_API_KEY', 'ANTHROPIC_API_KEY'] as const;
export function collectClaudeDesktopForwardedEnv(source: NodeJS.ProcessEnv): Record<string, string> {
const captured: Record<string, string> = {};
for (const [key, value] of Object.entries(source)) {
if (value === undefined || value === '') continue;
if (key.startsWith('KTX_') || (CLAUDE_DESKTOP_FORWARDED_ENV_KEYS as readonly string[]).includes(key)) {
captured[key] = value;
}
}
return captured;
}
function claudeDesktopMcpEntry(input: {
launcherPath: string;
projectDir: string;
env?: NodeJS.ProcessEnv;
}): Record<string, unknown> {
const captured = collectClaudeDesktopForwardedEnv(input.env ?? process.env);
return {
command: input.launcherPath,
args: ['--project-dir', input.projectDir, 'mcp', 'stdio'],
...(Object.keys(captured).length > 0 ? { env: captured } : {}),
};
}
async function installMcpClientConfig(input: { async function installMcpClientConfig(input: {
projectDir: string; projectDir: string;
target: KtxAgentTarget; target: KtxAgentTarget;
scope: KtxAgentScope; scope: KtxAgentScope;
}): Promise<KtxMcpClientInstallResult> { }): Promise<KtxMcpClientInstallResult> {
const endpoint = await resolveMcpEndpoint(input.projectDir);
const entries: InstallEntry[] = []; const entries: InstallEntry[] = [];
const snippets: string[] = []; const snippets: string[] = [];
const notices: string[] = []; const notices: string[] = [];
if (input.target === 'claude-desktop') {
const config = claudeDesktopConfigPath();
const launcherPath = claudeDesktopLauncherPath(input.projectDir);
await writeJsonKey(
config.path,
config.jsonPath,
claudeDesktopMcpEntry({ launcherPath, projectDir: input.projectDir }),
);
entries.push({ kind: 'json-key', path: config.path, jsonPath: config.jsonPath });
return { entries, snippets, notices };
}
const endpoint = await resolveMcpEndpoint(input.projectDir);
if (!endpoint.running) { if (!endpoint.running) {
notices.push('Run `ktx mcp start` to enable the configured KTX MCP server.'); notices.push('Run `ktx mcp start` to enable the configured KTX MCP server.');
} }
@ -213,74 +271,141 @@ async function installMcpClientConfig(input: {
} else if (input.target === 'codex') { } else if (input.target === 'codex') {
snippets.push(`Codex MCP snippet for ~/.codex/config.toml:\n${codexSnippet(endpoint)}`); snippets.push(`Codex MCP snippet for ~/.codex/config.toml:\n${codexSnippet(endpoint)}`);
} else if (input.target === 'opencode') { } else if (input.target === 'opencode') {
const path = input.scope === 'global' ? '~/.config/opencode/opencode.json' : relative(input.projectDir, join(input.projectDir, 'opencode.json')); const path =
input.scope === 'global'
? '~/.config/opencode/opencode.json'
: relative(input.projectDir, join(input.projectDir, 'opencode.json'));
snippets.push(`opencode MCP snippet for ${path}:\n${opencodeSnippet(endpoint)}`); snippets.push(`opencode MCP snippet for ${path}:\n${opencodeSnippet(endpoint)}`);
} else if (input.target === 'universal') {
snippets.push(universalMcpSnippet(endpoint));
} }
return { entries, snippets, notices }; return { entries, snippets, notices };
} }
function plannedMcpJsonEntries(input: {
projectDir: string;
target: KtxAgentTarget;
scope: KtxAgentScope;
}): InstallEntry[] {
if (input.target === 'claude-code') {
const config = claudeConfigPath(input.projectDir, input.scope);
return [{ kind: 'json-key', path: config.path, jsonPath: config.jsonPath }];
}
if (input.target === 'claude-desktop') {
const config = claudeDesktopConfigPath();
return [{ kind: 'json-key', path: config.path, jsonPath: config.jsonPath }];
}
if (input.target === 'cursor') {
const config = cursorConfigPath(input.projectDir, input.scope);
return [{ kind: 'json-key', path: config.path, jsonPath: config.jsonPath }];
}
return [];
}
export function agentInstallManifestPath(projectDir: string): string { export function agentInstallManifestPath(projectDir: string): string {
return join(resolve(projectDir), '.ktx/agents/install-manifest.json'); return join(resolve(projectDir), '.ktx/agents/install-manifest.json');
} }
function claudeDesktopPluginPath(projectDir: string): string {
return join(resolve(projectDir), '.ktx/agents/claude/ktx-plugin.zip');
}
function claudeDesktopLauncherPath(projectDir: string): string {
return join(resolve(projectDir), '.ktx/agents/claude/ktx-plugin-runner.sh');
}
export function plannedKtxAgentFiles(input: { export function plannedKtxAgentFiles(input: {
projectDir: string; projectDir: string;
target: KtxAgentTarget; target: KtxAgentTarget;
scope: KtxAgentScope; scope: KtxAgentScope;
mode: KtxAgentInstallMode; mode: KtxAgentInstallMode;
}): InstallEntry[] { }): InstallEntry[] {
const withAdminCli = input.mode === 'mcp-cli';
if (input.scope === 'global') { if (input.scope === 'global') {
if (input.target === 'claude-code') { if (input.target === 'claude-code') {
const home = process.env.HOME ?? ''; const home = process.env.HOME ?? '';
return [ return [
{ kind: 'file', path: join(home, '.claude/skills/ktx/SKILL.md'), role: 'skill' as const }, { kind: 'file', path: join(home, '.claude/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' as const },
{ kind: 'file', path: join(home, '.claude/skills/ktx-research/SKILL.md'), role: 'research-skill' as const }, ...(withAdminCli
{ kind: 'file', path: join(home, '.claude/rules/ktx.md'), role: 'rule' as const }, ? [
{ kind: 'file' as const, path: join(home, '.claude/skills/ktx/SKILL.md'), role: 'skill' as const },
{ kind: 'file' as const, path: join(home, '.claude/rules/ktx.md'), role: 'rule' as const },
]
: []),
]; ];
} }
if (input.target === 'codex') { if (input.target === 'codex') {
const codexHome = process.env.CODEX_HOME ?? join(process.env.HOME ?? '', '.codex'); const codexHome = process.env.CODEX_HOME ?? join(process.env.HOME ?? '', '.codex');
return [ return [
{ kind: 'file', path: join(codexHome, 'skills/ktx/SKILL.md'), role: 'skill' as const }, { kind: 'file', path: join(codexHome, 'skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' as const },
{ kind: 'file', path: join(codexHome, 'skills/ktx-research/SKILL.md'), role: 'research-skill' as const }, ...(withAdminCli
{ kind: 'file', path: join(codexHome, 'instructions/ktx.md'), role: 'rule' as const }, ? [
{ kind: 'file' as const, path: join(codexHome, 'skills/ktx/SKILL.md'), role: 'skill' as const },
{ kind: 'file' as const, path: join(codexHome, 'instructions/ktx.md'), role: 'rule' as const },
]
: []),
]; ];
} }
if (input.target === 'cursor' || input.target === 'opencode') { if (input.target === 'cursor' || input.target === 'opencode') {
return []; return [];
} }
if (input.target === 'claude-desktop') {
return [
{ kind: 'file', path: claudeDesktopLauncherPath(input.projectDir), role: 'launcher' as const },
{ kind: 'file', path: claudeDesktopPluginPath(input.projectDir), role: 'claude-plugin' as const },
];
}
throw new Error(`Global ${input.target} installation is not supported; omit --global.`); throw new Error(`Global ${input.target} installation is not supported; omit --global.`);
} }
const root = resolve(input.projectDir); const root = resolve(input.projectDir);
const analyticsEntries: Partial<Record<KtxAgentTarget, InstallEntry[]>> = {
'claude-code': [
{ kind: 'file', path: join(root, '.claude/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
],
codex: [
{ kind: 'file', path: join(root, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
],
cursor: [
{ kind: 'file', path: join(root, '.cursor/rules/ktx-analytics.mdc'), role: 'analytics-skill' },
],
opencode: [
{ kind: 'file', path: join(root, '.opencode/commands/ktx-analytics.md'), role: 'analytics-skill' },
],
universal: [
{ kind: 'file', path: join(root, '.agents/skills/ktx-analytics/SKILL.md'), role: 'analytics-skill' },
],
'claude-desktop': [],
};
const cliEntries: Partial<Record<KtxAgentTarget, InstallEntry[]>> = { const cliEntries: Partial<Record<KtxAgentTarget, InstallEntry[]>> = {
'claude-code': [ 'claude-code': [
{ kind: 'file', path: join(root, '.claude/skills/ktx/SKILL.md'), role: 'skill' }, { kind: 'file', path: join(root, '.claude/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(root, '.claude/skills/ktx-research/SKILL.md'), role: 'research-skill' },
], ],
codex: [ codex: [
{ kind: 'file', path: join(root, '.agents/skills/ktx/SKILL.md'), role: 'skill' }, { kind: 'file', path: join(root, '.agents/skills/ktx/SKILL.md'), role: 'skill' },
{ kind: 'file', path: join(root, '.agents/skills/ktx-research/SKILL.md'), role: 'research-skill' },
], ],
cursor: [ cursor: [
{ kind: 'file', path: join(root, '.cursor/rules/ktx.mdc') }, { kind: 'file', path: join(root, '.cursor/rules/ktx.mdc') },
{ kind: 'file', path: join(root, '.cursor/rules/ktx-research.mdc'), role: 'research-skill' },
], ],
opencode: [ opencode: [
{ kind: 'file', path: join(root, '.opencode/commands/ktx.md') }, { kind: 'file', path: join(root, '.opencode/commands/ktx.md') },
{ kind: 'file', path: join(root, '.opencode/commands/ktx-research.md'), role: 'research-skill' },
], ],
universal: [ universal: [
{ kind: 'file', path: join(root, '.agents/skills/ktx/SKILL.md') }, { kind: 'file', path: join(root, '.agents/skills/ktx/SKILL.md') },
{ kind: 'file', path: join(root, '.agents/skills/ktx-research/SKILL.md'), role: 'research-skill' },
], ],
'claude-desktop': [],
}; };
const ruleEntries: Partial<Record<KtxAgentTarget, InstallEntry>> = { const ruleEntries: Partial<Record<KtxAgentTarget, InstallEntry>> = {
'claude-code': { kind: 'file', path: join(root, '.claude/rules/ktx.md'), role: 'rule' }, 'claude-code': { kind: 'file', path: join(root, '.claude/rules/ktx.md'), role: 'rule' },
codex: { kind: 'file', path: join(root, '.codex/instructions/ktx.md'), role: 'rule' }, codex: { kind: 'file', path: join(root, '.codex/instructions/ktx.md'), role: 'rule' },
}; };
return [...(cliEntries[input.target] ?? []), ruleEntries[input.target]].filter( return [
...(analyticsEntries[input.target] ?? []),
...(withAdminCli ? (cliEntries[input.target] ?? []) : []),
...(withAdminCli ? [ruleEntries[input.target]] : []),
].filter(
(entry): entry is InstallEntry => entry !== undefined, (entry): entry is InstallEntry => entry !== undefined,
); );
} }
@ -292,8 +417,8 @@ function ktxCliLauncher(): KtxCliLauncher {
}; };
} }
async function readResearchSkillContent(): Promise<string> { async function readAnalyticsSkillContent(): Promise<string> {
const path = fileURLToPath(new URL('./skills/research/SKILL.md', import.meta.url)); const path = fileURLToPath(new URL('./skills/analytics/SKILL.md', import.meta.url));
const content = await readFile(path, 'utf-8'); const content = await readFile(path, 'utf-8');
return content.endsWith('\n') ? content : `${content}\n`; return content.endsWith('\n') ? content : `${content}\n`;
} }
@ -305,6 +430,10 @@ function shellQuote(value: string): string {
return `'${value.replaceAll("'", "'\\''")}'`; return `'${value.replaceAll("'", "'\\''")}'`;
} }
function shellScriptQuote(value: string): string {
return `'${value.replaceAll("'", "'\\''")}'`;
}
function ktxCommandLine(launcher: KtxCliLauncher, args: string[]): string { function ktxCommandLine(launcher: KtxCliLauncher, args: string[]): string {
return [launcher.command, ...launcher.args, ...args].map(shellQuote).join(' '); return [launcher.command, ...launcher.args, ...args].map(shellQuote).join(' ');
} }
@ -320,11 +449,14 @@ function cliInstructionContent(input: { projectDir: string; launcher: KtxCliLaun
'', '',
'# KTX Local Context', '# KTX Local Context',
'', '',
'This is an admin/developer CLI helper. End-user data agents should use the KTX MCP tools when available.',
'',
`Use this project with \`--project-dir ${input.projectDir}\`.`, `Use this project with \`--project-dir ${input.projectDir}\`.`,
'Commands are pinned to the local KTX CLI path that created this file, so agents do not need `ktx` in PATH.', 'Commands are pinned to the local KTX CLI path that created this file, so agents do not need `ktx` in PATH.',
'If the CLI path no longer exists after moving this checkout or reinstalling KTX, rerun `ktx setup --agents`.', 'If the CLI path no longer exists after moving this checkout or reinstalling KTX, rerun `ktx setup --agents`.',
'', '',
'Agents must not print secrets, credential references, environment variable values, or file contents from `.ktx/secrets`.', 'Agents must not print secrets, credential references, environment variable values, or file contents from ' +
'`.ktx/secrets`.',
'', '',
'Available commands:', 'Available commands:',
'', '',
@ -352,9 +484,132 @@ function cliInstructionContent(input: { projectDir: string; launcher: KtxCliLaun
].join('\n'); ].join('\n');
} }
function claudePluginJsonContent(): string {
return `${JSON.stringify(
{
name: 'ktx',
version: '0.0.0-local',
description: 'KTX analytics workflow guidance and local MCP tools.',
},
null,
2,
)}\n`;
}
function claudePluginVersionContent(): string {
return `${JSON.stringify({ version: '0.0.0-local' }, null, 2)}\n`;
}
function claudePluginSetupContent(input: { projectDir: string; withAdminCli: boolean }): string {
return [
'# KTX Claude Plugin',
'',
'Install this plugin ZIP from Claude Desktop to load the KTX analytics skill.',
'',
`KTX project: \`${input.projectDir}\``,
'',
'Included:',
'',
'- `ktx-analytics` skill for the MCP analytics workflow',
...(input.withAdminCli ? ['- `ktx` admin CLI skill for KTX maintenance commands'] : []),
'',
'The KTX MCP server is registered separately in `claude_desktop_config.json` by `ktx setup` and runs as a local stdio child of Claude Desktop — no daemon to start.',
'',
'If this checkout or project directory moves, rerun `ktx setup --agents` and reinstall the regenerated plugin.',
'',
].join('\n');
}
function claudePluginLauncherContent(input: { launcher: KtxCliLauncher }): string {
const binPath = input.launcher.args[0];
if (!binPath) {
throw new Error('Expected KTX CLI launcher to include a bin path.');
}
const candidates = [
input.launcher.command,
'/opt/homebrew/bin/node',
'/usr/local/bin/node',
'/usr/bin/node',
];
return [
'#!/bin/sh',
'set -eu',
'',
`KTX_CLI_BIN=${shellScriptQuote(binPath)}`,
'',
'run_with_node() {',
' node_bin=$1',
' shift',
' exec "$node_bin" "$KTX_CLI_BIN" "$@"',
'}',
'',
'if [ -n "${KTX_NODE:-}" ] && [ -x "${KTX_NODE:-}" ]; then',
' run_with_node "$KTX_NODE" "$@"',
'fi',
'',
'if [ -x "$HOME/.volta/bin/node" ]; then',
' run_with_node "$HOME/.volta/bin/node" "$@"',
'fi',
'',
...candidates.map((candidate) =>
[
`if [ -x ${shellScriptQuote(candidate)} ]; then`,
` run_with_node ${shellScriptQuote(candidate)} "$@"`,
'fi',
].join('\n'),
),
'',
'for candidate in "$HOME"/.nvm/versions/node/*/bin/node; do',
' if [ -x "$candidate" ]; then',
' run_with_node "$candidate" "$@"',
' fi',
'done',
'',
'if command -v node >/dev/null 2>&1; then',
' run_with_node "$(command -v node)" "$@"',
'fi',
'',
'echo "KTX plugin could not find Node.js. Set KTX_NODE to a Node executable and reinstall the plugin." >&2',
'exit 127',
'',
].join('\n');
}
async function writeClaudeDesktopPlugin(input: {
projectDir: string;
path: string;
mode: KtxAgentInstallMode;
launcher: KtxCliLauncher;
}): Promise<void> {
const withAdminCli = input.mode === 'mcp-cli';
const files: Record<string, Uint8Array> = {
'.claude-plugin/plugin.json': strToU8(claudePluginJsonContent()),
'version.json': strToU8(claudePluginVersionContent()),
'skills/ktx-analytics/SKILL.md': strToU8(await readAnalyticsSkillContent()),
'SETUP.md': strToU8(claudePluginSetupContent({ projectDir: input.projectDir, withAdminCli })),
};
if (withAdminCli) {
files['skills/ktx/SKILL.md'] = strToU8(
cliInstructionContent({ projectDir: input.projectDir, launcher: input.launcher }),
);
}
await mkdir(dirname(input.path), { recursive: true });
await writeFile(input.path, Buffer.from(zipSync(files)));
}
async function writeClaudeDesktopLauncher(input: {
path: string;
launcher: KtxCliLauncher;
}): Promise<void> {
await mkdir(dirname(input.path), { recursive: true });
await writeFile(input.path, claudePluginLauncherContent({ launcher: input.launcher }), 'utf-8');
await chmod(input.path, 0o755);
}
function ruleInstructionContent(input: { projectDir: string }): string { function ruleInstructionContent(input: { projectDir: string }): string {
return [ return [
`Use the \`ktx\` CLI to query local semantic context and wiki knowledge for this project (\`--project-dir ${input.projectDir}\`).`, `Use the \`ktx\` CLI to query local semantic context and wiki knowledge for this project ` +
`(\`--project-dir ${input.projectDir}\`).`,
'', '',
'Use when the user asks about data schemas, metrics, dimensions, database structure, or wants to run SQL queries.', 'Use when the user asks about data schemas, metrics, dimensions, database structure, or wants to run SQL queries.',
'', '',
@ -390,7 +645,9 @@ async function writeManifest(projectDir: string, manifest: KtxAgentInstallManife
} }
function entryKey(entry: InstallEntry): string { function entryKey(entry: InstallEntry): string {
return entry.kind === 'json-key' ? `${entry.kind}:${entry.path}:${entry.jsonPath.join('.')}` : `${entry.kind}:${entry.path}`; return entry.kind === 'json-key'
? `${entry.kind}:${entry.path}:${entry.jsonPath.join('.')}`
: `${entry.kind}:${entry.path}`;
} }
function mergeManifest( function mergeManifest(
@ -455,6 +712,7 @@ function createPromptAdapter(): KtxSetupAgentsPromptAdapter {
const targetDisplayNames: Record<KtxAgentTarget, string> = { const targetDisplayNames: Record<KtxAgentTarget, string> = {
'claude-code': 'Claude Code', 'claude-code': 'Claude Code',
'claude-desktop': 'Claude Desktop',
codex: 'Codex', codex: 'Codex',
cursor: 'Cursor', cursor: 'Cursor',
opencode: 'OpenCode', opencode: 'OpenCode',
@ -463,12 +721,25 @@ const targetDisplayNames: Record<KtxAgentTarget, string> = {
const fileEntryLabels: Record<KtxAgentTarget, string> = { const fileEntryLabels: Record<KtxAgentTarget, string> = {
'claude-code': 'Skill installed', 'claude-code': 'Skill installed',
'claude-desktop': 'Skill installed',
codex: 'Skill installed', codex: 'Skill installed',
cursor: 'Rule installed', cursor: 'Rule installed',
opencode: 'Command installed', opencode: 'Command installed',
universal: 'Skill installed', universal: 'Skill installed',
}; };
function mcpEntryLabel(entry: Extract<InstallEntry, { kind: 'json-key' }>): string {
return `MCP config installed — connects client agents to KTX MCP tools (${entry.jsonPath.join('.')})`;
}
function targetSupportsGlobalScope(target: KtxAgentTarget): boolean {
return target === 'claude-code' || target === 'codex';
}
function effectiveInstallScope(target: KtxAgentTarget, requestedScope: KtxAgentScope): KtxAgentScope {
return target === 'claude-desktop' ? 'global' : requestedScope;
}
export function formatInstallSummary( export function formatInstallSummary(
installs: Array<{ target: KtxAgentTarget; scope: KtxAgentScope; mode: KtxAgentInstallMode }>, installs: Array<{ target: KtxAgentTarget; scope: KtxAgentScope; mode: KtxAgentInstallMode }>,
entries: InstallEntry[], entries: InstallEntry[],
@ -486,11 +757,21 @@ export function formatInstallSummary(
entries.filter((entry) => entry.kind === 'file' && plannedFilePaths.has(entry.path)), entries.filter((entry) => entry.kind === 'file' && plannedFilePaths.has(entry.path)),
); );
} }
const mcpEntriesByTarget = new Map<KtxAgentTarget, InstallEntry[]>();
for (const install of installs) {
const plannedMcpKeys = new Set(plannedMcpJsonEntries({ projectDir, ...install }).map(entryKey));
mcpEntriesByTarget.set(
install.target,
entries.filter((entry) => entry.kind === 'json-key' && plannedMcpKeys.has(entryKey(entry))),
);
}
const fileHints: Record<string, string> = { const fileHints: Record<string, string> = {
skill: 'teaches your agent which KTX commands to run', skill: 'teaches admin agents which KTX CLI commands to run',
rule: 'tells your agent when to use KTX', rule: 'tells admin agents when to use KTX CLI',
'research-skill': 'teaches your agent the KTX MCP research workflow', 'analytics-skill': 'teaches your agent the KTX MCP analytics workflow',
'claude-plugin': 'bundles KTX skills for Claude Desktop (MCP server is registered in claude_desktop_config.json)',
launcher: 'runs the local KTX CLI with an available Node.js for Claude Desktop',
}; };
const lines: string[] = []; const lines: string[] = [];
@ -498,16 +779,34 @@ export function formatInstallSummary(
const targetEntries = entriesByTarget.get(install.target) ?? []; const targetEntries = entriesByTarget.get(install.target) ?? [];
lines.push(` ${targetDisplayNames[install.target]}`); lines.push(` ${targetDisplayNames[install.target]}`);
for (const entry of targetEntries) { for (const entry of targetEntries) {
const displayPath =
install.scope === 'global' ? entry.path : relative(projectDir, entry.path);
if (entry.kind === 'file') { if (entry.kind === 'file') {
const isRule = entry.role === 'rule' || fileEntryLabels[install.target] === 'Rule installed'; const isRule = entry.role === 'rule' || (!entry.role && fileEntryLabels[install.target] === 'Rule installed');
const label = entry.role === 'research-skill' ? 'Research skill installed' : isRule ? 'Rule installed' : fileEntryLabels[install.target]; const label =
entry.role === 'analytics-skill'
? 'Analytics skill installed'
: entry.role === 'claude-plugin'
? 'Claude plugin generated'
: entry.role === 'launcher'
? 'Launcher installed'
: isRule
? 'Rule installed'
: fileEntryLabels[install.target];
const hint = fileHints[isRule ? 'rule' : (entry.role ?? 'skill')] ?? ''; const hint = fileHints[isRule ? 'rule' : (entry.role ?? 'skill')] ?? '';
lines.push(` + ${label}${hint}`); lines.push(` + ${label}${hint}`);
lines.push(` ${displayPath}`); if (entry.role !== 'claude-plugin') {
const displayPath =
install.scope === 'global' ? entry.path : relative(projectDir, entry.path);
lines.push(` ${displayPath}`);
}
} }
} }
for (const entry of mcpEntriesByTarget
.get(install.target)
?.filter((entry): entry is Extract<InstallEntry, { kind: 'json-key' }> => entry.kind === 'json-key') ?? []) {
const displayPath = install.scope === 'global' ? entry.path : relative(projectDir, entry.path);
lines.push(` + ${mcpEntryLabel(entry)}`);
lines.push(` ${displayPath}`);
}
} }
return lines.join('\n'); return lines.join('\n');
} }
@ -522,11 +821,24 @@ async function installTarget(input: {
const launcher = ktxCliLauncher(); const launcher = ktxCliLauncher();
for (const entry of entries) { for (const entry of entries) {
if (entry.kind !== 'file') continue; if (entry.kind !== 'file') continue;
if (entry.role === 'launcher') {
await writeClaudeDesktopLauncher({ path: entry.path, launcher });
continue;
}
if (entry.role === 'claude-plugin') {
await writeClaudeDesktopPlugin({
projectDir: input.projectDir,
path: entry.path,
mode: input.mode,
launcher,
});
continue;
}
const content = const content =
entry.role === 'rule' entry.role === 'rule'
? ruleInstructionContent({ projectDir: input.projectDir }) ? ruleInstructionContent({ projectDir: input.projectDir })
: entry.role === 'research-skill' : entry.role === 'analytics-skill'
? await readResearchSkillContent() ? await readAnalyticsSkillContent()
: cliInstructionContent({ projectDir: input.projectDir, launcher }); : cliInstructionContent({ projectDir: input.projectDir, launcher });
await mkdir(dirname(entry.path), { recursive: true }); await mkdir(dirname(entry.path), { recursive: true });
await writeFile(entry.path, content, 'utf-8'); await writeFile(entry.path, content, 'utf-8');
@ -558,14 +870,13 @@ export async function runKtxSetupAgentsStep(
args.inputMode === 'disabled' args.inputMode === 'disabled'
? args.mode ? args.mode
: ((await prompts.select({ : ((await prompts.select({
message: 'How should agents use this KTX project?', message: 'How should client agents connect to this KTX project?',
options: [ options: [
{ value: 'cli', label: 'CLI tools and skills' }, { value: 'mcp', label: 'MCP tools + analytics skill' },
{ value: 'skip', label: 'Skip' }, { value: 'mcp-cli', label: 'MCP tools + analytics skill + admin CLI skill' },
], ],
})) as KtxAgentInstallMode | 'skip' | 'back'); })) as KtxAgentInstallMode | 'back');
if (mode === 'back') return { status: 'skipped', projectDir: args.projectDir }; if (mode === 'back') return { status: 'skipped', projectDir: args.projectDir };
if (mode === 'skip') return { status: 'skipped', projectDir: args.projectDir };
const targets = const targets =
args.target !== undefined args.target !== undefined
@ -576,6 +887,7 @@ export async function runKtxSetupAgentsStep(
message: withMultiselectNavigation('Which agent targets should KTX install?'), message: withMultiselectNavigation('Which agent targets should KTX install?'),
options: [ options: [
{ value: 'claude-code', label: 'Claude Code' }, { value: 'claude-code', label: 'Claude Code' },
{ value: 'claude-desktop', label: 'Claude Desktop' },
{ value: 'codex', label: 'Codex' }, { value: 'codex', label: 'Codex' },
{ value: 'cursor', label: 'Cursor' }, { value: 'cursor', label: 'Cursor' },
{ value: 'opencode', label: 'OpenCode' }, { value: 'opencode', label: 'OpenCode' },
@ -589,26 +901,80 @@ export async function runKtxSetupAgentsStep(
return { status: 'missing-input', projectDir: args.projectDir }; return { status: 'missing-input', projectDir: args.projectDir };
} }
const installs = targets.map((target) => ({ target, scope: args.scope, mode })); const scopeTargets = targets.filter((target) => target !== 'claude-desktop');
const selectedScope =
args.inputMode !== 'disabled' &&
args.scope === 'project' &&
scopeTargets.length > 0 &&
scopeTargets.every(targetSupportsGlobalScope)
? ((await prompts.select({
message: 'Where should KTX install supported agent config?',
options: [
{ value: 'project', label: 'Project' },
{ value: 'global', label: 'Global' },
],
})) as KtxAgentScope | 'back')
: args.scope;
if (selectedScope === 'back') return { status: 'back', projectDir: args.projectDir };
const installs = targets.map((target) => ({ target, scope: effectiveInstallScope(target, selectedScope), mode }));
const entries: InstallEntry[] = []; const entries: InstallEntry[] = [];
const snippets: string[] = []; const snippets: string[] = [];
const notices = new Set<string>(); const notices = new Set<string>();
let claudeDesktopTutorial: string | undefined;
try { try {
for (const install of installs) { for (const install of installs) {
entries.push(...(await installTarget({ projectDir: args.projectDir, ...install }))); const targetEntries = await installTarget({ projectDir: args.projectDir, ...install });
const mcpResult = await installMcpClientConfig({ projectDir: args.projectDir, target: install.target, scope: install.scope }); entries.push(...targetEntries);
const mcpResult = await installMcpClientConfig({
projectDir: args.projectDir,
target: install.target,
scope: install.scope,
});
entries.push(...mcpResult.entries); entries.push(...mcpResult.entries);
for (const snippet of mcpResult.snippets) snippets.push(snippet); for (const snippet of mcpResult.snippets) snippets.push(snippet);
for (const notice of mcpResult.notices) notices.add(notice); for (const notice of mcpResult.notices) notices.add(notice);
if (install.target === 'claude-desktop') {
const pluginEntry = targetEntries.find(
(entry): entry is Extract<InstallEntry, { kind: 'file' }> =>
entry.kind === 'file' && entry.role === 'claude-plugin',
);
const pluginPath = pluginEntry?.path ?? '';
const configPath = claudeDesktopConfigPath().path;
claudeDesktopTutorial = [
`${green('✓')} ${bold('KTX MCP server registered')}`,
` ${dim(configPath)}`,
'',
bold('1. Restart Claude Desktop'),
' Quit and reopen so it picks up the new MCP server.',
'',
bold('2. Install the KTX plugin'),
' Open Claude Desktop → Settings → Plugins and install from file:',
` 📦 ${dim(pluginPath)}`,
].join('\n');
}
} }
await writeManifest(args.projectDir, mergeManifest(args.projectDir, await readKtxAgentInstallManifest(args.projectDir), installs, entries)); await writeManifest(
args.projectDir,
mergeManifest(args.projectDir, await readKtxAgentInstallManifest(args.projectDir), installs, entries),
);
await markAgentsComplete(args.projectDir); await markAgentsComplete(args.projectDir);
io.stdout.write(`\nAgent integration complete\n\n${formatInstallSummary(installs, entries, args.projectDir)}\n`); const setupUi = createKtxSetupUiAdapter();
for (const snippet of snippets) { setupUi.note(
io.stdout.write(`\n${snippet}\n`); formatInstallSummary(installs, entries, args.projectDir),
'Agent integration complete',
io,
);
if (claudeDesktopTutorial) {
setupUi.note(claudeDesktopTutorial, 'Finish Claude Desktop setup', io, {
format: (line) => line,
});
} }
for (const notice of notices) { const nextStepBlocks: string[] = [];
io.stdout.write(`\n${notice}\n`); for (const notice of notices) nextStepBlocks.push(notice);
for (const snippet of snippets) nextStepBlocks.push(snippet);
if (nextStepBlocks.length > 0) {
setupUi.note(nextStepBlocks.join('\n\n'), 'Next steps', io, { format: bold });
} }
return { status: 'ready', projectDir: args.projectDir, installs }; return { status: 'ready', projectDir: args.projectDir, installs };
} catch (error) { } catch (error) {

View file

@ -184,7 +184,7 @@ describe('runDemoTour', () => {
const mockAgents = vi.fn().mockResolvedValue({ const mockAgents = vi.fn().mockResolvedValue({
status: 'ready', status: 'ready',
projectDir: '/tmp/test', projectDir: '/tmp/test',
installs: [{ target: 'claude-code', scope: 'project', mode: 'cli' }], installs: [{ target: 'claude-code', scope: 'project', mode: 'mcp-cli' }],
} satisfies KtxSetupAgentsResult); } satisfies KtxSetupAgentsResult);
const navigation = vi.fn().mockResolvedValue('forward'); const navigation = vi.fn().mockResolvedValue('forward');

View file

@ -375,7 +375,7 @@ export async function runDemoTour(
yes: false, yes: false,
agents: true, agents: true,
scope: 'project', scope: 'project',
mode: 'cli', mode: 'mcp-cli',
skipAgents: false, skipAgents: false,
}, },
io, io,

View file

@ -138,9 +138,13 @@ export function createKtxSetupPromptAdapter(options: KtxSetupPromptAdapterOption
}; };
} }
interface KtxSetupNoteOptions {
format?: (line: string) => string;
}
export interface KtxSetupUiAdapter { export interface KtxSetupUiAdapter {
intro(title: string, io: KtxCliIo): void; intro(title: string, io: KtxCliIo): void;
note(message: string, title: string, io: KtxCliIo): void; note(message: string, title: string, io: KtxCliIo, options?: KtxSetupNoteOptions): void;
} }
function isWritableTtyOutput(output: KtxCliIo['stdout']): output is KtxCliIo['stdout'] & Writable { function isWritableTtyOutput(output: KtxCliIo['stdout']): output is KtxCliIo['stdout'] & Writable {
@ -160,9 +164,12 @@ export function createKtxSetupUiAdapter(): KtxSetupUiAdapter {
} }
io.stdout.write(`${title}\n`); io.stdout.write(`${title}\n`);
}, },
note(message, title, io) { note(message, title, io, options) {
if (isWritableTtyOutput(io.stdout)) { if (isWritableTtyOutput(io.stdout)) {
note(message, title, { output: io.stdout }); note(message, title, {
output: io.stdout,
...(options?.format ? { format: options.format } : {}),
});
return; return;
} }
io.stdout.write(`\n${title}:\n`); io.stdout.write(`\n${title}:\n`);

View file

@ -232,7 +232,10 @@ describe('setup status', () => {
version: 1, version: 1,
projectDir: tempDir, projectDir: tempDir,
installedAt: '2026-05-07T00:00:00.000Z', installedAt: '2026-05-07T00:00:00.000Z',
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }], installs: [
{ target: 'codex', scope: 'project', mode: 'mcp' },
{ target: 'codex', scope: 'project', mode: 'mcp-cli' },
],
entries: [], entries: [],
}, },
null, null,
@ -1051,6 +1054,53 @@ describe('setup status', () => {
); );
}); });
it('auto-installs the managed runtime by default during setup', async () => {
const io = makeIo();
const embeddings = vi.fn(async () => ({ status: 'ready' as const, projectDir: tempDir }));
const context = vi.fn(async () => ({ status: 'failed' as const, projectDir: tempDir }));
await expect(
runKtxSetup(
{
command: 'run',
projectDir: tempDir,
mode: 'new',
agents: false,
agentScope: 'project',
skipAgents: true,
inputMode: 'auto',
yes: false,
cliVersion: '0.2.0',
skipLlm: true,
skipEmbeddings: false,
databaseSchemas: [],
skipDatabases: true,
skipSources: true,
},
io.io,
{
embeddings,
context,
},
),
).resolves.toBe(1);
expect(embeddings).toHaveBeenCalledWith(
expect.objectContaining({
cliVersion: '0.2.0',
runtimeInstallPolicy: 'auto',
}),
io.io,
);
expect(context).toHaveBeenCalledWith(
expect.objectContaining({
cliVersion: '0.2.0',
runtimeInstallPolicy: 'auto',
}),
io.io,
);
});
it('lets Back from embedding setup return to the model step instead of exiting', async () => { it('lets Back from embedding setup return to the model step instead of exiting', async () => {
const testIo = makeIo(); const testIo = makeIo();
const modelResults = [ const modelResults = [
@ -1467,7 +1517,7 @@ describe('setup status', () => {
return { return {
status: 'ready', status: 'ready',
projectDir: tempDir, projectDir: tempDir,
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }], installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
}; };
}, },
}, },
@ -1519,7 +1569,7 @@ describe('setup status', () => {
agents: async () => ({ agents: async () => ({
status: 'ready', status: 'ready',
projectDir: tempDir, projectDir: tempDir,
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }], installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
}), }),
}, },
), ),
@ -1570,7 +1620,7 @@ describe('setup status', () => {
return { return {
status: 'ready', status: 'ready',
projectDir: tempDir, projectDir: tempDir,
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }], installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
}; };
}, },
}, },
@ -1585,7 +1635,7 @@ describe('setup status', () => {
const agents = vi.fn(async () => ({ const agents = vi.fn(async () => ({
status: 'ready' as const, status: 'ready' as const,
projectDir: tempDir, projectDir: tempDir,
installs: [{ target: 'codex' as const, scope: 'project' as const, mode: 'cli' as const }], installs: [{ target: 'codex' as const, scope: 'project' as const, mode: 'mcp-cli' as const }],
})); }));
await writeFile(join(tempDir, 'ktx.yaml'), ['connections: {}', ''].join('\n'), 'utf-8'); await writeFile(join(tempDir, 'ktx.yaml'), ['connections: {}', ''].join('\n'), 'utf-8');
@ -1654,7 +1704,7 @@ describe('setup status', () => {
version: 1, version: 1,
projectDir: tempDir, projectDir: tempDir,
installedAt: '2026-05-07T00:00:00.000Z', installedAt: '2026-05-07T00:00:00.000Z',
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }], installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
entries: [], entries: [],
}, },
null, null,
@ -1717,7 +1767,7 @@ describe('setup status', () => {
return { return {
status: 'ready', status: 'ready',
projectDir: tempDir, projectDir: tempDir,
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }], installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
}; };
}, },
}, },
@ -1809,7 +1859,7 @@ describe('setup status', () => {
return { return {
status: 'ready', status: 'ready',
projectDir: tempDir, projectDir: tempDir,
installs: [{ target: 'codex', scope: 'project', mode: 'cli' }], installs: [{ target: 'codex', scope: 'project', mode: 'mcp-cli' }],
}; };
}, },
}, },
@ -1826,7 +1876,7 @@ describe('setup status', () => {
const agents = vi.fn(async () => ({ const agents = vi.fn(async () => ({
status: 'ready' as const, status: 'ready' as const,
projectDir: tempDir, projectDir: tempDir,
installs: [{ target: 'universal' as const, scope: 'project' as const, mode: 'cli' as const }], installs: [{ target: 'universal' as const, scope: 'project' as const, mode: 'mcp-cli' as const }],
})); }));
await expect( await expect(

View file

@ -307,12 +307,15 @@ export async function readKtxSetupStatus(projectDir: string): Promise<KtxSetupSt
const databaseIds = project.config.setup?.database_connection_ids ?? Object.keys(project.config.connections); const databaseIds = project.config.setup?.database_connection_ids ?? Object.keys(project.config.connections);
const databasesComplete = completedSteps.includes('databases'); const databasesComplete = completedSteps.includes('databases');
const manifest = await readKtxAgentInstallManifest(resolvedProjectDir); const manifest = await readKtxAgentInstallManifest(resolvedProjectDir);
const agents = const agentMap = new Map<string, { target: string; scope: string; ready: boolean }>();
manifest?.installs.map((install) => ({ for (const install of manifest?.installs ?? []) {
agentMap.set(`${install.target}:${install.scope}`, {
target: install.target, target: install.target,
scope: install.scope, scope: install.scope,
ready: true, ready: true,
})) ?? []; });
}
const agents = [...agentMap.values()];
return { return {
project: { path: resolvedProjectDir, ready: true, name: basename(project.projectDir) || project.projectDir }, project: { path: resolvedProjectDir, ready: true, name: basename(project.projectDir) || project.projectDir },
@ -413,10 +416,7 @@ function writeContextNotReadyForAgents(projectDir: string, io: KtxCliIo): void {
} }
function setupRuntimeInstallPolicy(args: Extract<KtxSetupArgs, { command: 'run' }>): 'prompt' | 'auto' | 'never' { function setupRuntimeInstallPolicy(args: Extract<KtxSetupArgs, { command: 'run' }>): 'prompt' | 'auto' | 'never' {
if (args.yes) { return args.inputMode === 'disabled' && !args.yes ? 'never' : 'auto';
return 'auto';
}
return args.inputMode === 'disabled' ? 'never' : 'prompt';
} }
async function commitSetupConfigChanges(projectDir: string): Promise<void> { async function commitSetupConfigChanges(projectDir: string): Promise<void> {
@ -662,7 +662,7 @@ async function runKtxSetupInner(args: KtxSetupArgs, io: KtxCliIo, deps: KtxSetup
agents: true, agents: true,
...(args.target ? { target: args.target } : {}), ...(args.target ? { target: args.target } : {}),
scope: args.agentScope ?? 'project', scope: args.agentScope ?? 'project',
mode: 'cli', mode: 'mcp',
skipAgents: false, skipAgents: false,
}, },
io, io,
@ -707,16 +707,21 @@ async function runKtxSetupInner(args: KtxSetupArgs, io: KtxCliIo, deps: KtxSetup
await commitSetupConfigChanges(projectResult.projectDir); await commitSetupConfigChanges(projectResult.projectDir);
const status = await readKtxSetupStatus(projectResult.projectDir); const status = await readKtxSetupStatus(projectResult.projectDir);
io.stdout.write(formatKtxSetupStatus(status)); const focusedOnAgents = args.agents || entryAction === 'agents';
setupUi.note( if (!focusedOnAgents) {
formatSetupNextStepLines({ setupUi.note(formatKtxSetupStatus(status).trimEnd(), 'Project status', io, {
setupReady: setupStatusReady(status), format: (line) => line,
hasContextTargets: setupHasContextTargets(status), });
contextReady: setupContextReady(status), setupUi.note(
agentIntegrationReady: status.agents.some((agent) => agent.ready), formatSetupNextStepLines({
}).join('\n'), setupReady: setupStatusReady(status),
'What you can do next', hasContextTargets: setupHasContextTargets(status),
io, contextReady: setupContextReady(status),
); agentIntegrationReady: status.agents.some((agent) => agent.ready),
}).join('\n'),
'What you can do next',
io,
);
}
return 0; return 0;
} }

View file

@ -0,0 +1,62 @@
---
name: ktx-analytics
description: Use when answering a question that needs data from a KTX-connected database - investigating, analyzing, "how many", "show me", "what's the breakdown of", finding records by value, exploring tables, comparing periods, explaining metrics, or any data-analysis request. Triggers even when the user does not say "analytics"; if the answer requires querying a configured KTX connection, this skill applies.
---
# KTX Analytics Workflow
You have access to KTX MCP tools for data discovery, semantic-layer analysis, raw read-only SQL, wiki context, and memory ingest. Follow this workflow.
<workflow>
1. **Discover** - call `discover_data` first to see what exists across wiki pages, semantic-layer sources, metrics, dimensions, raw tables, and columns. Returns refs only.
2. **Inspect top hits in parallel** - for each promising ref:
- `kind: 'wiki'` -> `wiki_read`
- `kind: 'sl_source'`, `kind: 'sl_measure'`, or `kind: 'sl_dimension'` -> `sl_read_source`
- `kind: 'table'` or `kind: 'column'` -> `entity_details`
3. **Resolve business values** - if the user named a value such as "Acme Corp", "enterprise", or "status=shipped", call `dictionary_search` to find which column holds it.
4. **Plan the analysis** - identify the grain, metrics, dimensions, filters, time window, and expected row limits before querying.
5. **Query** -
- Prefer `sl_query` when the semantic layer covers the question.
- Use `sql_execution` only for questions the semantic layer does not cover.
6. **Validate and explain** - sanity-check totals, filters, null handling, and time zones. State the source tables or semantic-layer objects used.
7. **Capture durable learnings** - call `memory_ingest` whenever a turn produces something worth remembering (business rules, metric definitions, schema gotchas, recurring findings) **or** whenever the user asks you to remember something. Pass markdown in `content` including any source context the memory agent should weigh. Each call is a feedback loop; better notes today mean smarter `discover_data` and `wiki_search` results tomorrow.
</workflow>
<rules>
- Always run `discover_data` before writing SQL. Do not guess table names.
- Prefer the semantic layer over raw SQL when both can answer the question; measures are the source of truth.
- Read entity details before writing SQL against an unfamiliar table. Do not assume column names.
- Treat `sql_execution` as read-only. Writes are rejected by the server.
- Validate value mentions with `dictionary_search` instead of guessing case or spelling. Treat a `dictionary_search` miss as non-authoritative. The index is built from profile-sampled values, so a missing value may simply have been outside the sample. Follow up with `sql_execution` against the most plausible columns before concluding the value is absent.
- When `connection_list` shows multiple connections, pass an explicit `connectionId` to every tool that takes one and where user intent pins a specific warehouse. Required: `entity_details`, `sl_read_source`, and `sql_execution`. Required when user intent is warehouse-specific, including wording like "in our warehouse" or "this warehouse": `memory_ingest`; without `connectionId`, the memory agent cannot update the semantic layer and the knowledge lands as wiki-only. Pass `connectionId` when intent pins a warehouse, otherwise omit for unscoped discovery: `sl_query`, `discover_data`, and `dictionary_search`. Never pass `connectionId` to `connection_list`, `wiki_search`, `wiki_read`, or `memory_ingest_status`. If intent is ambiguous for a required-or-scoped tool, ask the user which warehouse before calling.
- Show compact result tables for small outputs. For broad results, summarize the top findings and mention the applied limit.
- Ask a concise clarification only when the metric, date range, entity, or grain is genuinely ambiguous and cannot be inferred from context.
</rules>
<examples>
**Input:** "How many orders did Acme Corp place last month?"
**Workflow:**
1. `dictionary_search({ values: ["Acme Corp"] })` finds `customers.name`.
2. `discover_data({ query: "orders customer monthly" })` finds an orders semantic-layer source.
3. `sl_read_source({ connectionId: "warehouse", sourceName: "orders_facts" })` confirms the source grain, measures, and dimensions.
4. `sl_query({ connectionId: "warehouse", measures: ["order_count"], filters: ["customer_name = 'Acme Corp'"] })` answers through the semantic layer.
5. `memory_ingest({ connectionId: "warehouse", content: "Acme Corp order analysis used orders_facts.order_count filtered by customers.name = 'Acme Corp'. Source: current analysis turn." })` captures the durable finding.
---
**Input:** "What columns does the events table have?"
**Workflow:**
1. `discover_data({ query: "events table" })` returns a `table` ref.
2. `entity_details({ connectionId: "warehouse", entities: [{ table: "analytics.events" }] })` returns columns, types, and foreign keys.
3. Answer directly. No query is needed.
---
**Input:** "Heads up: ARR is always reported in cents in our warehouse."
**Workflow:**
1. If multiple connections exist, call `connection_list` and identify the warehouse the user means. Ask if ambiguous.
2. `memory_ingest({ connectionId: "warehouse", content: "ARR is reported in cents (not dollars) in this warehouse. Multiply by 0.01 for dollar amounts. Source: user clarification." })` remembers the warehouse-specific rule without running an analysis turn.
</examples>

View file

@ -1,49 +0,0 @@
---
name: ktx-research
description: Use when answering a question that needs data from a KTX-connected database - investigating, analyzing, "how many", "show me", "what's the breakdown of", finding records by value, exploring tables, comparing periods, or any data-investigation request. Triggers even when the user does not say "research"; if the answer requires querying a configured KTX connection, this skill applies.
---
# KTX Research Workflow
You have access to KTX MCP tools for investigating data. Follow this workflow.
<workflow>
1. **Discover** - call `discover_data` first to see what exists across wiki, semantic-layer sources, and raw tables. Returns refs only.
2. **Inspect top hits in parallel** - for each promising ref:
- `kind: 'wiki'` -> `wiki_read`
- `kind: 'sl_source'`, `kind: 'sl_measure'`, or `kind: 'sl_dimension'` -> `sl_read_source`
- `kind: 'table'` or `kind: 'column'` -> `entity_details`
3. **Resolve literals** - if the user named a value such as "Acme Corp" or "status=shipped", call `dictionary_search` to find which column holds it.
4. **Query** -
- Prefer `sl_query` when the semantic layer covers the question.
- Use `sql_execution` only for questions the semantic layer does not cover.
5. **Capture learnings** - at the end of the turn, call `memory_capture` so future turns benefit. Skip when the answer carries no durable knowledge.
</workflow>
<rules>
- Always run `discover_data` before writing SQL. Do not guess table names.
- Prefer the semantic layer over raw SQL when both can answer the question; measures are the source of truth.
- Read entity details before writing SQL against an unfamiliar table. Do not assume column names.
- Treat `sql_execution` as read-only. Writes are rejected by the server.
- Validate value mentions with `dictionary_search` instead of guessing case or spelling. Treat a `dictionary_search` miss as non-authoritative. The index is built from profile-sampled values, so a missing value may simply have been outside the sample. Follow up with `sql_execution` against the most plausible columns before concluding the value is absent.
</rules>
<examples>
**Input:** "How many orders did Acme Corp place last month?"
**Workflow:**
1. `dictionary_search({ values: ["Acme Corp"] })` finds `customers.name`.
2. `discover_data({ query: "orders customer monthly" })` finds an orders semantic-layer source.
3. `sl_read_source({ connectionId: "warehouse", sourceName: "orders_facts" })` confirms the source grain, measures, and dimensions.
4. `sl_query({ connectionId: "warehouse", measures: ["order_count"], filters: ["customer_name = 'Acme Corp'"] })` answers through the semantic layer.
5. `memory_capture({ userMessage, assistantMessage })` captures the durable finding.
---
**Input:** "What columns does the events table have?"
**Workflow:**
1. `discover_data({ query: "events table" })` returns a `table` ref.
2. `entity_details({ connectionId: "warehouse", entities: [{ table: "analytics.events" }] })` returns columns, types, and foreign keys.
3. Answer directly. No query is needed.
</examples>

View file

@ -1,7 +1,7 @@
import { describe, expect, it, vi } from 'vitest'; import { describe, expect, it, vi } from 'vitest';
import type { MemoryCaptureStatus } from '@ktx/context/memory'; import type { MemoryIngestStatus } from '@ktx/context/memory';
import type { KtxLocalProject } from '@ktx/context/project'; import type { KtxLocalProject } from '@ktx/context/project';
import { runKtxTextIngest, type TextMemoryCapturePort } from './text-ingest.js'; import { runKtxTextIngest, type TextMemoryIngestPort } from './text-ingest.js';
function makeIo(options: { isTTY?: boolean } = {}) { function makeIo(options: { isTTY?: boolean } = {}) {
let stdout = ''; let stdout = '';
@ -25,18 +25,18 @@ function makeIo(options: { isTTY?: boolean } = {}) {
}; };
} }
function fakeCapture( function fakeIngest(
options: { options: {
failRunIds?: Set<string>; failRunIds?: Set<string>;
missingStatusRunIds?: Set<string>; missingStatusRunIds?: Set<string>;
events?: string[]; events?: string[];
} = {}, } = {},
): TextMemoryCapturePort { ): TextMemoryIngestPort {
let next = 1; let next = 1;
return { return {
capture: vi.fn(async () => { ingest: vi.fn(async () => {
const runId = `run-${next++}`; const runId = `run-${next++}`;
options.events?.push(`capture:${runId}`); options.events?.push(`ingest:${runId}`);
return { runId }; return { runId };
}), }),
waitForRun: vi.fn(async (runId: string) => { waitForRun: vi.fn(async (runId: string) => {
@ -51,26 +51,26 @@ function fakeCapture(
return { return {
runId, runId,
status: 'error', status: 'error',
stage: 'capturing', stage: 'ingesting',
done: true, done: true,
captured: { wiki: [], sl: [], xrefs: [] }, captured: { wiki: [], sl: [], xrefs: [] },
error: `${runId} failed`, error: `${runId} failed`,
commitHash: null, commitHash: null,
skillsLoaded: [], skillsLoaded: [],
signalDetected: false, signalDetected: false,
} satisfies MemoryCaptureStatus; } satisfies MemoryIngestStatus;
} }
return { return {
runId, runId,
status: 'done', status: 'done',
stage: 'capturing', stage: 'ingesting',
done: true, done: true,
captured: { wiki: [`wiki-${runId}`], sl: [`sl-${runId}`], xrefs: [] }, captured: { wiki: [`wiki-${runId}`], sl: [`sl-${runId}`], xrefs: [] },
error: null, error: null,
commitHash: `commit-${runId}`, commitHash: `commit-${runId}`,
skillsLoaded: ['wiki_capture', 'sl'], skillsLoaded: ['wiki_capture', 'sl'],
signalDetected: true, signalDetected: true,
} satisfies MemoryCaptureStatus; } satisfies MemoryIngestStatus;
}), }),
}; };
} }
@ -80,11 +80,11 @@ function fakeProject(projectDir = '/tmp/project'): KtxLocalProject {
} }
describe('runKtxTextIngest', () => { describe('runKtxTextIngest', () => {
it('captures repeated inline text sequentially with generated internal chat ids', async () => { it('ingests repeated inline text sequentially with generated internal chat ids', async () => {
const io = makeIo(); const io = makeIo();
const events: string[] = []; const events: string[] = [];
const capture = fakeCapture({ events }); const ingest = fakeIngest({ events });
const createMemoryCapture = vi.fn(() => capture); const createMemoryIngest = vi.fn(() => ingest);
await expect( await expect(
runKtxTextIngest( runKtxTextIngest(
@ -99,14 +99,14 @@ describe('runKtxTextIngest', () => {
io.io, io.io,
{ {
loadProject: vi.fn(async () => fakeProject()), loadProject: vi.fn(async () => fakeProject()),
createMemoryCapture, createMemoryIngest,
now: () => 1_700_000_000_000, now: () => 1_700_000_000_000,
}, },
), ),
).resolves.toBe(0); ).resolves.toBe(0);
expect(createMemoryCapture).toHaveBeenCalledWith({ projectDir: '/tmp/project' }); expect(createMemoryIngest).toHaveBeenCalledWith({ projectDir: '/tmp/project' });
expect(capture.capture).toHaveBeenNthCalledWith( expect(ingest.ingest).toHaveBeenNthCalledWith(
1, 1,
expect.objectContaining({ expect.objectContaining({
userId: 'local-cli', userId: 'local-cli',
@ -116,7 +116,7 @@ describe('runKtxTextIngest', () => {
sourceType: 'external_ingest', sourceType: 'external_ingest',
}), }),
); );
expect(capture.capture).toHaveBeenNthCalledWith( expect(ingest.ingest).toHaveBeenNthCalledWith(
2, 2,
expect.objectContaining({ expect.objectContaining({
chatId: 'cli-text-ingest-1700000000000-2', chatId: 'cli-text-ingest-1700000000000-2',
@ -124,8 +124,8 @@ describe('runKtxTextIngest', () => {
assistantMessage: 'Orders are completed purchases.', assistantMessage: 'Orders are completed purchases.',
}), }),
); );
expect(capture.capture).not.toHaveBeenCalledWith(expect.objectContaining({ connectionId: expect.anything() })); expect(ingest.ingest).not.toHaveBeenCalledWith(expect.objectContaining({ connectionId: expect.anything() }));
expect(events).toEqual(['capture:run-1', 'wait:run-1', 'status:run-1', 'capture:run-2', 'wait:run-2', 'status:run-2']); expect(events).toEqual(['ingest:run-1', 'wait:run-1', 'status:run-1', 'ingest:run-2', 'wait:run-2', 'status:run-2']);
expect(JSON.parse(io.stdout())).toMatchObject({ expect(JSON.parse(io.stdout())).toMatchObject({
status: 'done', status: 'done',
results: [ results: [
@ -147,7 +147,7 @@ describe('runKtxTextIngest', () => {
it('loads files and stdin as batch items and passes a global connection id', async () => { it('loads files and stdin as batch items and passes a global connection id', async () => {
const io = makeIo(); const io = makeIo();
const capture = fakeCapture(); const ingest = fakeIngest();
await expect( await expect(
runKtxTextIngest( runKtxTextIngest(
@ -163,7 +163,7 @@ describe('runKtxTextIngest', () => {
io.io, io.io,
{ {
loadProject: vi.fn(async () => fakeProject()), loadProject: vi.fn(async () => fakeProject()),
createMemoryCapture: vi.fn(() => capture), createMemoryIngest: vi.fn(() => ingest),
readFile: vi.fn(async (path) => `file:${path}`), readFile: vi.fn(async (path) => `file:${path}`),
readStdin: vi.fn(async () => 'stdin content'), readStdin: vi.fn(async () => 'stdin content'),
now: () => 10, now: () => 10,
@ -171,7 +171,7 @@ describe('runKtxTextIngest', () => {
), ),
).resolves.toBe(0); ).resolves.toBe(0);
expect(capture.capture).toHaveBeenNthCalledWith( expect(ingest.ingest).toHaveBeenNthCalledWith(
1, 1,
expect.objectContaining({ expect.objectContaining({
connectionId: 'warehouse', connectionId: 'warehouse',
@ -180,7 +180,7 @@ describe('runKtxTextIngest', () => {
assistantMessage: 'file:/tmp/docs/revenue.md', assistantMessage: 'file:/tmp/docs/revenue.md',
}), }),
); );
expect(capture.capture).toHaveBeenNthCalledWith( expect(ingest.ingest).toHaveBeenNthCalledWith(
2, 2,
expect.objectContaining({ expect.objectContaining({
connectionId: 'warehouse', connectionId: 'warehouse',
@ -194,9 +194,9 @@ describe('runKtxTextIngest', () => {
expect(io.stdout()).toContain('stdin'); expect(io.stdout()).toContain('stdin');
}); });
it('uses bounded inline text previews as labels in plain output and capture metadata', async () => { it('uses bounded inline text previews as labels in plain output and ingest metadata', async () => {
const io = makeIo(); const io = makeIo();
const capture = fakeCapture(); const ingest = fakeIngest();
const longText = `This inline note is intentionally long ${'x'.repeat(120)}`; const longText = `This inline note is intentionally long ${'x'.repeat(120)}`;
await expect( await expect(
@ -212,7 +212,7 @@ describe('runKtxTextIngest', () => {
io.io, io.io,
{ {
loadProject: vi.fn(async () => fakeProject()), loadProject: vi.fn(async () => fakeProject()),
createMemoryCapture: vi.fn(() => capture), createMemoryIngest: vi.fn(() => ingest),
now: () => 10, now: () => 10,
}, },
), ),
@ -225,19 +225,19 @@ describe('runKtxTextIngest', () => {
expect(output).not.toContain('text-1'); expect(output).not.toContain('text-1');
expect(output).not.toContain(longText); expect(output).not.toContain(longText);
expect(capture.capture).toHaveBeenNthCalledWith( expect(ingest.ingest).toHaveBeenNthCalledWith(
1, 1,
expect.objectContaining({ expect.objectContaining({
userMessage: 'Ingest external text artifact "remember to call me Andrey" into KTX memory.', userMessage: 'Ingest external text artifact "remember to call me Andrey" into KTX memory.',
}), }),
); );
expect(capture.capture).toHaveBeenNthCalledWith( expect(ingest.ingest).toHaveBeenNthCalledWith(
2, 2,
expect.objectContaining({ expect.objectContaining({
userMessage: 'Ingest external text artifact "first line second line" into KTX memory.', userMessage: 'Ingest external text artifact "first line second line" into KTX memory.',
}), }),
); );
expect(capture.capture).toHaveBeenNthCalledWith( expect(ingest.ingest).toHaveBeenNthCalledWith(
3, 3,
expect.objectContaining({ expect.objectContaining({
userMessage: 'Ingest external text artifact "This inline note is intentionally long xxxxxxxx..." into KTX memory.', userMessage: 'Ingest external text artifact "This inline note is intentionally long xxxxxxxx..." into KTX memory.',
@ -247,7 +247,7 @@ describe('runKtxTextIngest', () => {
it('continues after an item failure by default and stops when failFast is set', async () => { it('continues after an item failure by default and stops when failFast is set', async () => {
const continueIo = makeIo(); const continueIo = makeIo();
const continueCapture = fakeCapture({ failRunIds: new Set(['run-1']) }); const continueIngest = fakeIngest({ failRunIds: new Set(['run-1']) });
await expect( await expect(
runKtxTextIngest( runKtxTextIngest(
@ -262,12 +262,12 @@ describe('runKtxTextIngest', () => {
continueIo.io, continueIo.io,
{ {
loadProject: vi.fn(async () => fakeProject()), loadProject: vi.fn(async () => fakeProject()),
createMemoryCapture: vi.fn(() => continueCapture), createMemoryIngest: vi.fn(() => continueIngest),
}, },
), ),
).resolves.toBe(1); ).resolves.toBe(1);
expect(continueCapture.capture).toHaveBeenCalledTimes(2); expect(continueIngest.ingest).toHaveBeenCalledTimes(2);
expect(JSON.parse(continueIo.stdout())).toMatchObject({ expect(JSON.parse(continueIo.stdout())).toMatchObject({
status: 'failed', status: 'failed',
results: [ results: [
@ -277,7 +277,7 @@ describe('runKtxTextIngest', () => {
}); });
const failFastIo = makeIo(); const failFastIo = makeIo();
const failFastCapture = fakeCapture({ failRunIds: new Set(['run-1']) }); const failFastIngest = fakeIngest({ failRunIds: new Set(['run-1']) });
await expect( await expect(
runKtxTextIngest( runKtxTextIngest(
@ -292,12 +292,12 @@ describe('runKtxTextIngest', () => {
failFastIo.io, failFastIo.io,
{ {
loadProject: vi.fn(async () => fakeProject()), loadProject: vi.fn(async () => fakeProject()),
createMemoryCapture: vi.fn(() => failFastCapture), createMemoryIngest: vi.fn(() => failFastIngest),
}, },
), ),
).resolves.toBe(1); ).resolves.toBe(1);
expect(failFastCapture.capture).toHaveBeenCalledTimes(1); expect(failFastIngest.ingest).toHaveBeenCalledTimes(1);
expect(JSON.parse(failFastIo.stdout()).results).toHaveLength(1); expect(JSON.parse(failFastIo.stdout()).results).toHaveLength(1);
}); });
@ -314,7 +314,7 @@ describe('runKtxTextIngest', () => {
failFast: false, failFast: false,
}, },
noInputIo.io, noInputIo.io,
{ loadProject: vi.fn(), createMemoryCapture: vi.fn() }, { loadProject: vi.fn(), createMemoryIngest: vi.fn() },
), ),
).resolves.toBe(1); ).resolves.toBe(1);
expect(noInputIo.stderr()).toContain('Provide at least one text item'); expect(noInputIo.stderr()).toContain('Provide at least one text item');
@ -331,7 +331,7 @@ describe('runKtxTextIngest', () => {
failFast: false, failFast: false,
}, },
emptyIo.io, emptyIo.io,
{ loadProject: vi.fn(), createMemoryCapture: vi.fn() }, { loadProject: vi.fn(), createMemoryIngest: vi.fn() },
), ),
).resolves.toBe(1); ).resolves.toBe(1);
expect(emptyIo.stderr()).toContain('Text item "text-1" is empty'); expect(emptyIo.stderr()).toContain('Text item "text-1" is empty');

View file

@ -1,6 +1,6 @@
import { readFile as fsReadFile } from 'node:fs/promises'; import { readFile as fsReadFile } from 'node:fs/promises';
import { basename, resolve } from 'node:path'; import { basename, resolve } from 'node:path';
import { createLocalProjectMemoryCapture, type MemoryAgentInput, type MemoryCaptureStatus } from '@ktx/context/memory'; import { createLocalProjectMemoryIngest, type MemoryAgentInput, type MemoryIngestStatus } from '@ktx/context/memory';
import { loadKtxProject, type KtxLocalProject } from '@ktx/context/project'; import { loadKtxProject, type KtxLocalProject } from '@ktx/context/project';
import type { KtxCliIo } from './cli-runtime.js'; import type { KtxCliIo } from './cli-runtime.js';
import { createRepainter, initViewState, renderContextBuildView, type ContextBuildTargetState } from './context-build-view.js'; import { createRepainter, initViewState, renderContextBuildView, type ContextBuildTargetState } from './context-build-view.js';
@ -17,10 +17,10 @@ export interface KtxTextIngestArgs {
failFast: boolean; failFast: boolean;
} }
export interface TextMemoryCapturePort { export interface TextMemoryIngestPort {
capture(input: MemoryAgentInput): Promise<{ runId: string }>; ingest(input: MemoryAgentInput): Promise<{ runId: string }>;
waitForRun(runId: string): Promise<void>; waitForRun(runId: string): Promise<void>;
status(runId: string): Promise<MemoryCaptureStatus | null>; status(runId: string): Promise<MemoryIngestStatus | null>;
} }
interface TextIngestItem { interface TextIngestItem {
@ -32,14 +32,14 @@ interface TextIngestResult {
label: string; label: string;
runId: string | null; runId: string | null;
status: 'done' | 'error'; status: 'done' | 'error';
captured: MemoryCaptureStatus['captured']; captured: MemoryIngestStatus['captured'];
commitHash: string | null; commitHash: string | null;
error: string | null; error: string | null;
} }
export interface KtxTextIngestDeps { export interface KtxTextIngestDeps {
loadProject?: (options: { projectDir: string }) => Promise<KtxLocalProject>; loadProject?: (options: { projectDir: string }) => Promise<KtxLocalProject>;
createMemoryCapture?: (project: KtxLocalProject) => TextMemoryCapturePort; createMemoryIngest?: (project: KtxLocalProject) => TextMemoryIngestPort;
readFile?: (path: string) => Promise<string>; readFile?: (path: string) => Promise<string>;
readStdin?: () => Promise<string>; readStdin?: () => Promise<string>;
now?: () => number; now?: () => number;
@ -48,8 +48,8 @@ export interface KtxTextIngestDeps {
const INLINE_TEXT_LABEL_MAX_LENGTH = 50; const INLINE_TEXT_LABEL_MAX_LENGTH = 50;
const ANSI_ESCAPE_PATTERN = /\x1B\[[0-?]*[ -/]*[@-~]/g; const ANSI_ESCAPE_PATTERN = /\x1B\[[0-?]*[ -/]*[@-~]/g;
function defaultCreateMemoryCapture(project: KtxLocalProject): TextMemoryCapturePort { function defaultCreateMemoryIngest(project: KtxLocalProject): TextMemoryIngestPort {
return createLocalProjectMemoryCapture(project); return createLocalProjectMemoryIngest(project);
} }
async function defaultReadStdin(): Promise<string> { async function defaultReadStdin(): Promise<string> {
@ -65,7 +65,7 @@ async function defaultReadFile(path: string): Promise<string> {
return await fsReadFile(path, 'utf-8'); return await fsReadFile(path, 'utf-8');
} }
function emptyCaptured(): MemoryCaptureStatus['captured'] { function emptyCaptured(): MemoryIngestStatus['captured'] {
return { wiki: [], sl: [], xrefs: [] }; return { wiki: [], sl: [], xrefs: [] };
} }
@ -182,7 +182,7 @@ function renderTextIngestView(state: ReturnType<typeof initViewState>, styled: b
}); });
} }
function summarizeCaptured(captured: MemoryCaptureStatus['captured']): string { function summarizeCaptured(captured: MemoryIngestStatus['captured']): string {
const parts = [ const parts = [
`wiki=${captured.wiki.length}`, `wiki=${captured.wiki.length}`,
`sl=${captured.sl.length}`, `sl=${captured.sl.length}`,
@ -191,7 +191,7 @@ function summarizeCaptured(captured: MemoryCaptureStatus['captured']): string {
return parts.join(', '); return parts.join(', ');
} }
function resultFromStatus(label: string, status: MemoryCaptureStatus): TextIngestResult { function resultFromStatus(label: string, status: MemoryIngestStatus): TextIngestResult {
return { return {
label, label,
runId: status.runId, runId: status.runId,
@ -251,7 +251,7 @@ export async function runKtxTextIngest(
} }
const project = await (deps.loadProject ?? loadKtxProject)({ projectDir: args.projectDir }); const project = await (deps.loadProject ?? loadKtxProject)({ projectDir: args.projectDir });
const memoryCapture = (deps.createMemoryCapture ?? defaultCreateMemoryCapture)(project); const memoryIngest = (deps.createMemoryIngest ?? defaultCreateMemoryIngest)(project);
const now = deps.now ?? (() => Date.now()); const now = deps.now ?? (() => Date.now());
const batchId = now(); const batchId = now();
const state = initViewState(items.map((item) => makeTarget(item.label))); const state = initViewState(items.map((item) => makeTarget(item.label)));
@ -292,7 +292,7 @@ export async function runKtxTextIngest(
let runId: string | null = null; let runId: string | null = null;
let result: TextIngestResult; let result: TextIngestResult;
try { try {
const captureInput: MemoryAgentInput = { const ingestInput: MemoryAgentInput = {
userId: args.userId, userId: args.userId,
chatId: `cli-text-ingest-${batchId}-${index + 1}`, chatId: `cli-text-ingest-${batchId}-${index + 1}`,
userMessage: `Ingest external text artifact ${artifactReference(item.label)} into KTX memory.`, userMessage: `Ingest external text artifact ${artifactReference(item.label)} into KTX memory.`,
@ -300,12 +300,12 @@ export async function runKtxTextIngest(
...(args.connectionId ? { connectionId: args.connectionId } : {}), ...(args.connectionId ? { connectionId: args.connectionId } : {}),
sourceType: 'external_ingest', sourceType: 'external_ingest',
}; };
const capture = await memoryCapture.capture(captureInput); const ingest = await memoryIngest.ingest(ingestInput);
runId = capture.runId; runId = ingest.runId;
await memoryCapture.waitForRun(runId); await memoryIngest.waitForRun(runId);
const status = await memoryCapture.status(runId); const status = await memoryIngest.status(runId);
if (!status) { if (!status) {
throw new Error(`Memory capture run "${runId}" was not found.`); throw new Error(`Memory ingest run "${runId}" was not found.`);
} }
result = resultFromStatus(item.label, status); result = resultFromStatus(item.label, status);
} catch (error) { } catch (error) {

View file

@ -2,7 +2,7 @@ import { request as httpRequest } from 'node:http';
import { request as httpsRequest } from 'node:https'; import { request as httpsRequest } from 'node:https';
import { URL } from 'node:url'; import { URL } from 'node:url';
import { spawn } from 'node:child_process'; import { spawn } from 'node:child_process';
import type { SemanticLayerQueryInput, SemanticLayerSource } from '../sl/index.js'; import type { ResolvedSemanticLayerSource, SemanticLayerQueryInput } from '../sl/types.js';
export interface KtxSemanticLayerComputeQueryResult { export interface KtxSemanticLayerComputeQueryResult {
sql: string; sql: string;
@ -54,13 +54,21 @@ export interface KtxSemanticLayerSourceGenerationResult {
} }
export interface KtxSemanticLayerComputePort { export interface KtxSemanticLayerComputePort {
/**
* Callers must pass sources sanitized through toResolvedWire. The Python
* daemon rejects authoring-only fields such as usage and inherits_columns_from.
*/
query(input: { query(input: {
sources: Array<Record<string, unknown> | SemanticLayerSource>; sources: ResolvedSemanticLayerSource[];
query: SemanticLayerQueryInput; query: SemanticLayerQueryInput;
dialect: string; dialect: string;
}): Promise<KtxSemanticLayerComputeQueryResult>; }): Promise<KtxSemanticLayerComputeQueryResult>;
/**
* Callers must pass sources sanitized through toResolvedWire. The Python
* daemon rejects authoring-only fields such as usage and inherits_columns_from.
*/
validateSources(input: { validateSources(input: {
sources: Array<Record<string, unknown> | SemanticLayerSource>; sources: ResolvedSemanticLayerSource[];
dialect: string; dialect: string;
recentlyTouched?: string[]; recentlyTouched?: string[];
}): Promise<KtxSemanticLayerComputeValidationResult>; }): Promise<KtxSemanticLayerComputeValidationResult>;

View file

@ -191,6 +191,36 @@ describe('local KTX embedding config', () => {
expect(resolveLocalKtxEmbeddingConfig(config, {})).toBeNull(); expect(resolveLocalKtxEmbeddingConfig(config, {})).toBeNull();
}); });
it('returns null when backend is openai but no apiKey is resolvable from env', () => {
const config: KtxProjectEmbeddingConfig = {
backend: 'openai',
model: 'text-embedding-3-small',
dimensions: 1536,
openai: { api_key: 'env:OPENAI_API_KEY' }, // pragma: allowlist secret
};
expect(resolveLocalKtxEmbeddingConfig(config, {})).toBeNull();
});
it('resolves openai embedding config from env', () => {
const config: KtxProjectEmbeddingConfig = {
backend: 'openai',
model: 'text-embedding-3-small',
dimensions: 1536,
openai: { api_key: 'env:OPENAI_API_KEY' }, // pragma: allowlist secret
};
expect(
resolveLocalKtxEmbeddingConfig(config, { OPENAI_API_KEY: 'sk-test' }), // pragma: allowlist secret
).toEqual({
backend: 'openai',
model: 'text-embedding-3-small',
dimensions: 1536,
openai: { apiKey: 'sk-test' }, // pragma: allowlist secret
batchSize: undefined,
});
});
it('constructs deterministic embeddings from the default project config', () => { it('constructs deterministic embeddings from the default project config', () => {
const createKtxEmbeddingProvider = vi.fn(() => ({}) as never); const createKtxEmbeddingProvider = vi.fn(() => ({}) as never);
const provider = createLocalKtxEmbeddingProviderFromConfig( const provider = createLocalKtxEmbeddingProviderFromConfig(

View file

@ -177,11 +177,23 @@ export function resolveLocalKtxEmbeddingConfig(
batchSize: config.batchSize, batchSize: config.batchSize,
}; };
} }
if (config.backend === 'openai') {
const openai = resolvedProviderConfig(config.openai, env);
if (!openai?.apiKey) {
return null;
}
return {
backend: config.backend,
model: config.model ?? 'deterministic',
dimensions: config.dimensions,
openai,
batchSize: config.batchSize,
};
}
return { return {
backend: config.backend, backend: config.backend,
model: config.model ?? 'deterministic', model: config.model ?? 'deterministic',
dimensions: config.dimensions, dimensions: config.dimensions,
...(resolvedProviderConfig(config.openai, env) ? { openai: resolvedProviderConfig(config.openai, env) } : {}),
...(config.sentenceTransformers ...(config.sentenceTransformers
? { ? {
sentenceTransformers: { sentenceTransformers: {

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -8,29 +8,18 @@ export type {
KtxDiscoverDataMcpPort, KtxDiscoverDataMcpPort,
KtxDictionarySearchMcpPort, KtxDictionarySearchMcpPort,
KtxEntityDetailsMcpPort, KtxEntityDetailsMcpPort,
KtxIngestDiffSummary,
KtxIngestMcpPort,
KtxIngestStatusResponse,
KtxIngestTriggerKind,
KtxIngestTriggerResponse,
KtxIngestWorkUnitSummary,
KtxKnowledgeMcpPort, KtxKnowledgeMcpPort,
KtxKnowledgePage, KtxKnowledgePage,
KtxKnowledgeSearchResponse, KtxKnowledgeSearchResponse,
KtxKnowledgeSearchResult, KtxKnowledgeSearchResult,
KtxKnowledgeWriteResponse,
KtxMcpContextPorts, KtxMcpContextPorts,
KtxMcpServerDeps, KtxMcpServerDeps,
KtxMcpServerLike, KtxMcpServerLike,
KtxMcpTextContent, KtxMcpTextContent,
KtxMcpToolResult, KtxMcpToolResult,
KtxMcpUserContext, KtxMcpUserContext,
KtxSemanticLayerListResponse,
KtxSemanticLayerMcpPort, KtxSemanticLayerMcpPort,
KtxSemanticLayerQueryResponse, KtxSemanticLayerQueryResponse,
KtxSemanticLayerReadResponse, KtxSemanticLayerReadResponse,
KtxSemanticLayerSourceSummary, MemoryIngestPort,
KtxSemanticLayerValidationResponse,
KtxSemanticLayerWriteResponse,
MemoryCapturePort,
} from './types.js'; } from './types.js';

File diff suppressed because it is too large Load diff

View file

@ -1,65 +1,19 @@
import YAML from 'yaml'; import { type KtxSqlQueryExecutorPort, localConnectionInfoFromConfig } from '../connections/index.js';
import {
type KtxSqlQueryExecutorPort,
localConnectionInfoFromConfig,
localConnectionTypeForConfig,
} from '../connections/index.js';
import type { KtxEmbeddingPort } from '../core/index.js'; import type { KtxEmbeddingPort } from '../core/index.js';
import type { KtxSemanticLayerComputePort } from '../daemon/index.js'; import type { KtxSemanticLayerComputePort } from '../daemon/index.js';
import {
createDefaultLocalIngestAdapters,
getLocalIngestStatus,
type IngestReportSnapshot,
ingestReportToMemoryFlowReplay,
type LocalIngestMcpOptions,
runLocalIngest,
runLocalMetabaseIngest,
} from '../ingest/index.js';
import { createLocalKtxEmbeddingProviderFromConfig, KtxIngestEmbeddingPortAdapter } from '../llm/index.js'; import { createLocalKtxEmbeddingProviderFromConfig, KtxIngestEmbeddingPortAdapter } from '../llm/index.js';
import type { KtxLocalProject } from '../project/index.js'; import type { KtxLocalProject } from '../project/index.js';
import { import { createKtxEntityDetailsService, type KtxScanConnector, type LocalScanMcpOptions } from '../scan/index.js';
createKtxEntityDetailsService,
getLocalScanReport,
getLocalScanStatus,
type KtxConnectionDriver,
type KtxScanConnector,
type KtxScanReport,
type LocalScanMcpOptions,
runLocalScan,
} from '../scan/index.js';
import { createKtxDiscoverDataService } from '../search/index.js'; import { createKtxDiscoverDataService } from '../search/index.js';
import type { SqlAnalysisDialect, SqlAnalysisPort } from '../sql-analysis/index.js'; import type { SqlAnalysisDialect, SqlAnalysisPort } from '../sql-analysis/index.js';
import { import { compileLocalSlQuery, createKtxDictionarySearchService } from '../sl/index.js';
compileLocalSlQuery, import { readLocalKnowledgePage, searchLocalKnowledgePages } from '../wiki/local-knowledge.js';
createKtxDictionarySearchService, import type { KtxMcpContextPorts, KtxMcpProgressCallback, KtxSqlExecutionResponse } from './types.js';
type LocalSlSourceSearchResult,
type LocalSlSourceSummary,
listLocalSlSources,
searchLocalSlSources,
sourceDefinitionSchema,
sourceOverlaySchema,
} from '../sl/index.js';
import { readLocalKnowledgePage, searchLocalKnowledgePages, writeLocalKnowledgePage } from '../wiki/local-knowledge.js';
import type {
KtxConnectionTestResponse,
KtxIngestStatusResponse,
KtxMcpContextPorts,
KtxScanArtifactListResponse,
KtxScanArtifactReadResponse,
KtxScanArtifactSummary,
KtxScanArtifactType,
KtxSqlExecutionResponse,
} from './types.js';
const LOCAL_AUTHOR = 'ktx';
const LOCAL_AUTHOR_EMAIL = 'ktx@example.com';
const SL_SHAPE_WARNING = 'Local stdio validation checks YAML shape only; Python semantic validation is not configured.';
interface CreateLocalProjectMcpContextPortsOptions { interface CreateLocalProjectMcpContextPortsOptions {
semanticLayerCompute?: KtxSemanticLayerComputePort; semanticLayerCompute?: KtxSemanticLayerComputePort;
queryExecutor?: KtxSqlQueryExecutorPort; queryExecutor?: KtxSqlQueryExecutorPort;
sqlAnalysis?: SqlAnalysisPort; sqlAnalysis?: SqlAnalysisPort;
localIngest?: LocalIngestMcpOptions;
localScan?: LocalScanMcpOptions; localScan?: LocalScanMcpOptions;
embeddingService?: KtxEmbeddingPort | null; embeddingService?: KtxEmbeddingPort | null;
} }
@ -115,284 +69,23 @@ function assertSafeSourceName(sourceName: string): string {
return assertSafePathToken('semantic-layer source name', sourceName); return assertSafePathToken('semantic-layer source name', sourceName);
} }
function normalizeScanDriver(driver: string | undefined): KtxConnectionDriver {
const normalized = (driver ?? '').toLowerCase();
if (
normalized === 'postgres' ||
normalized === 'postgresql' ||
normalized === 'sqlite' ||
normalized === 'sqlite3' ||
normalized === 'mysql' ||
normalized === 'clickhouse' ||
normalized === 'sqlserver' ||
normalized === 'bigquery' ||
normalized === 'snowflake'
) {
return normalized === 'sqlite3' ? 'sqlite' : normalized;
}
return 'postgres';
}
async function cleanupConnector(connector: KtxScanConnector | null): Promise<void> { async function cleanupConnector(connector: KtxScanConnector | null): Promise<void> {
if (connector?.cleanup) { if (connector?.cleanup) {
await connector.cleanup(); await connector.cleanup();
} }
} }
async function testLocalConnection(
project: KtxLocalProject,
options: CreateLocalProjectMcpContextPortsOptions,
connectionId: string,
): Promise<KtxConnectionTestResponse | null> {
const safeConnectionId = assertSafeConnectionId(connectionId);
const connection = project.config.connections[safeConnectionId];
if (!connection) {
return null;
}
const connectionType = localConnectionTypeForConfig(safeConnectionId, connection);
const createConnector = options.localScan?.createConnector;
if (!createConnector) {
return {
id: safeConnectionId,
connectionType,
ok: true,
tableCount: null,
message: 'Connection is configured; no native scan connector is available for live testing.',
warnings: ['ktx serve was not configured with a local scan connector factory.'],
};
}
let connector: KtxScanConnector | null = null;
try {
connector = await createConnector(safeConnectionId);
const snapshot = await connector.introspect(
{
connectionId: safeConnectionId,
driver: normalizeScanDriver(connection.driver),
mode: 'structural',
dryRun: true,
detectRelationships: false,
},
{ runId: `connection-test-${safeConnectionId}` },
);
return {
id: safeConnectionId,
connectionType,
ok: true,
tableCount: snapshot.tables.length,
message: 'Connection test passed.',
warnings: [],
};
} catch (error) {
return {
id: safeConnectionId,
connectionType,
ok: false,
tableCount: null,
message: error instanceof Error ? error.message : String(error),
warnings: [],
};
} finally {
await cleanupConnector(connector);
}
}
function scanArtifactType(path: string, report: KtxScanReport): KtxScanArtifactType {
if (path === report.artifactPaths.reportPath) {
return 'report';
}
if (report.artifactPaths.manifestShards.includes(path)) {
return 'manifest_shard';
}
if (report.artifactPaths.enrichmentArtifacts.includes(path)) {
return 'enrichment_artifact';
}
return 'raw_source';
}
async function artifactSize(project: KtxLocalProject, path: string): Promise<number | undefined> {
try {
const result = await project.fileStore.readFile(path);
return typeof result.size === 'number' ? result.size : undefined;
} catch {
return undefined;
}
}
async function listArtifactsForReport(
project: KtxLocalProject,
runId: string,
report: KtxScanReport,
): Promise<KtxScanArtifactListResponse> {
const paths = new Set<string>();
if (report.artifactPaths.rawSourcesDir) {
const listed = await project.fileStore.listFiles(report.artifactPaths.rawSourcesDir);
for (const file of listed.files) {
paths.add(file);
}
}
if (report.artifactPaths.reportPath) {
paths.add(report.artifactPaths.reportPath);
}
for (const path of report.artifactPaths.manifestShards) {
paths.add(path);
}
for (const path of report.artifactPaths.enrichmentArtifacts) {
paths.add(path);
}
const artifacts: KtxScanArtifactSummary[] = [];
for (const path of [...paths].sort()) {
const size = await artifactSize(project, path);
artifacts.push({
path,
type: scanArtifactType(path, report),
...(size === undefined ? {} : { size }),
});
}
return { runId, artifacts };
}
async function readScanArtifact(
project: KtxLocalProject,
runId: string,
path: string,
): Promise<KtxScanArtifactReadResponse | null> {
const report = await getLocalScanReport(project, runId);
if (!report) {
return null;
}
const listed = await listArtifactsForReport(project, runId, report);
const artifact = listed.artifacts.find((candidate) => candidate.path === path);
if (!artifact) {
return null;
}
const result = await project.fileStore.readFile(path);
return {
runId,
path,
type: artifact.type,
...(typeof result.size === 'number' ? { size: result.size } : {}),
content: result.content,
};
}
function slPath(connectionId: string, sourceName: string): string { function slPath(connectionId: string, sourceName: string): string {
return `semantic-layer/${assertSafeConnectionId(connectionId)}/${assertSafeSourceName(sourceName)}.yaml`; return `semantic-layer/${assertSafeConnectionId(connectionId)}/${assertSafeSourceName(sourceName)}.yaml`;
} }
function sourceNameFromPath(path: string): string {
return (
path
.split('/')
.at(-1)
?.replace(/\.ya?ml$/, '') ?? path
);
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
function parseYamlRecord(raw: string): Record<string, unknown> {
const parsed = YAML.parse(raw) as unknown;
if (!isRecord(parsed)) {
throw new Error('Semantic-layer source YAML must contain an object');
}
return parsed;
}
async function listSlPaths(project: KtxLocalProject, connectionId?: string): Promise<string[]> {
const root = connectionId ? `semantic-layer/${assertSafeConnectionId(connectionId)}` : 'semantic-layer';
const listed = await project.fileStore.listFiles(root);
return listed.files.filter((file) => file.endsWith('.yaml') || file.endsWith('.yml')).sort();
}
async function loadComputableSources(
project: KtxLocalProject,
connectionId: string,
): Promise<Record<string, unknown>[]> {
const paths = await listSlPaths(project, connectionId);
const sources: Record<string, unknown>[] = [];
for (const path of paths) {
const raw = await project.fileStore.readFile(path);
const source = parseYamlRecord(raw.content);
if (source.table || source.sql) {
sources.push(source);
}
}
return sources;
}
function validateSourceRecord(sourceName: string, source: Record<string, unknown>): string[] {
const namedSource = { ...source, name: typeof source.name === 'string' ? source.name : sourceName };
const definition = sourceDefinitionSchema.safeParse(namedSource);
if (definition.success) {
return [];
}
const overlay = sourceOverlaySchema.safeParse(namedSource);
if (overlay.success) {
return [];
}
return definition.error.issues.map((issue) => `${sourceName}: ${issue.path.join('.') || 'source'} ${issue.message}`);
}
function localIngestSourceDir(config: unknown): string | undefined {
if (!isRecord(config) || config.sourceDir === undefined) {
return undefined;
}
if (typeof config.sourceDir !== 'string' || config.sourceDir.trim().length === 0) {
throw new Error('Local ingest config sourceDir must be a non-empty string when provided');
}
return config.sourceDir;
}
function rawFileCountFromIngestReport(report: IngestReportSnapshot): number {
return new Set(report.body.workUnits.flatMap((workUnit) => workUnit.rawFiles)).size;
}
function hasSlSearchMetadata(
source: LocalSlSourceSummary | LocalSlSourceSearchResult,
): source is LocalSlSourceSearchResult {
return 'score' in source;
}
function statusFromIngestReport(report: IngestReportSnapshot): KtxIngestStatusResponse {
const failedWorkUnits = report.body.failedWorkUnits;
return {
runId: report.runId,
jobId: report.jobId,
reportId: report.id,
status: failedWorkUnits.length > 0 ? 'error' : 'done',
stage: 'done',
progress: 1,
errors: failedWorkUnits,
done: true,
adapter: report.sourceKey,
connectionId: report.connectionId,
sourceDir: null,
syncId: report.body.syncId,
startedAt: report.createdAt,
completedAt: report.createdAt,
previousRunId: null,
diffSummary: report.body.diffSummary,
workUnitCount: report.body.workUnits.length,
rawFileCount: rawFileCountFromIngestReport(report),
workUnits: report.body.workUnits.map((workUnit) => ({
unitKey: workUnit.unitKey,
rawFiles: [...workUnit.rawFiles],
peerFileIndex: [],
dependencyPaths: [],
})),
evictionDeletedRawPaths: [...report.body.evictionInputs],
};
}
async function executeValidatedReadOnlySql( async function executeValidatedReadOnlySql(
project: KtxLocalProject, project: KtxLocalProject,
options: CreateLocalProjectMcpContextPortsOptions, options: CreateLocalProjectMcpContextPortsOptions,
input: { connectionId: string; sql: string; maxRows: number }, input: { connectionId: string; sql: string; maxRows: number },
onProgress?: KtxMcpProgressCallback,
): Promise<KtxSqlExecutionResponse> { ): Promise<KtxSqlExecutionResponse> {
await onProgress?.({ progress: 0, message: 'Validating SQL' });
const connectionId = assertSafeConnectionId(input.connectionId); const connectionId = assertSafeConnectionId(input.connectionId);
const connection = project.config.connections[connectionId]; const connection = project.config.connections[connectionId];
if (!connection) { if (!connection) {
@ -416,6 +109,7 @@ async function executeValidatedReadOnlySql(
if (!connector.capabilities.readOnlySql || !connector.executeReadOnly) { if (!connector.capabilities.readOnlySql || !connector.executeReadOnly) {
throw new Error(`Connection "${connectionId}" does not support read-only SQL execution.`); throw new Error(`Connection "${connectionId}" does not support read-only SQL execution.`);
} }
await onProgress?.({ progress: 0.3, message: 'Executing' });
const result = await connector.executeReadOnly( const result = await connector.executeReadOnly(
{ {
connectionId, connectionId,
@ -424,12 +118,14 @@ async function executeValidatedReadOnlySql(
}, },
{ runId: 'mcp-sql-execution' }, { runId: 'mcp-sql-execution' },
); );
return { const response = {
headers: result.headers, headers: result.headers,
...(result.headerTypes ? { headerTypes: result.headerTypes } : {}), ...(result.headerTypes ? { headerTypes: result.headerTypes } : {}),
rows: result.rows, rows: result.rows,
rowCount: result.rowCount ?? result.rows.length, rowCount: result.rowCount ?? result.rows.length,
}; };
await onProgress?.({ progress: 1, message: `Fetched ${response.rowCount} rows` });
return response;
} finally { } finally {
await cleanupConnector(connector); await cleanupConnector(connector);
} }
@ -453,9 +149,6 @@ export function createLocalProjectMcpContextPorts(
) )
.sort((a, b) => a.id.localeCompare(b.id)); .sort((a, b) => a.id.localeCompare(b.id));
}, },
async test(input) {
return testLocalConnection(project, options, input.connectionId);
},
}, },
knowledge: { knowledge: {
async search(input) { async search(input) {
@ -495,58 +188,8 @@ export function createLocalProjectMcpContextPorts(
} }
: null; : null;
}, },
async write(input) {
const existing = await readLocalKnowledgePage(project, {
key: input.key,
userId: input.userId,
});
await writeLocalKnowledgePage(project, {
key: input.key,
scope: 'GLOBAL',
userId: input.userId,
summary: input.summary,
content: input.content,
tags: input.tags,
refs: input.refs,
slRefs: input.slRefs,
source: input.source,
intent: input.intent,
tables: input.tables,
representativeSql: input.representativeSql,
usage: input.usage,
fingerprints: input.fingerprints,
});
return { success: true, key: input.key, action: existing ? 'updated' : 'created' };
},
}, },
semanticLayer: { semanticLayer: {
async listSources(input) {
const listed: Array<LocalSlSourceSummary | LocalSlSourceSearchResult> = input.query
? await searchLocalSlSources(project, {
connectionId: input.connectionId,
query: input.query,
embeddingService,
})
: await listLocalSlSources(project, { connectionId: input.connectionId });
const sources = listed.map((source) => ({
connectionId: source.connectionId,
connectionName: source.connectionId,
name: source.name,
description: source.description,
columnCount: source.columnCount,
measureCount: source.measureCount,
joinCount: source.joinCount,
...(hasSlSearchMetadata(source) && source.frequencyTier ? { frequencyTier: source.frequencyTier } : {}),
...(hasSlSearchMetadata(source) && source.snippet ? { snippet: source.snippet } : {}),
...(hasSlSearchMetadata(source) ? { score: source.score } : {}),
...(hasSlSearchMetadata(source) && source.matchReasons ? { matchReasons: source.matchReasons } : {}),
...(hasSlSearchMetadata(source) && source.dictionaryMatches
? { dictionaryMatches: source.dictionaryMatches }
: {}),
...(hasSlSearchMetadata(source) && source.lanes ? { lanes: source.lanes } : {}),
}));
return { sources, totalSources: sources.length };
},
async readSource(input) { async readSource(input) {
const path = slPath(input.connectionId, input.sourceName); const path = slPath(input.connectionId, input.sourceName);
try { try {
@ -556,71 +199,9 @@ export function createLocalProjectMcpContextPorts(
return null; return null;
} }
}, },
async writeSource(input) { async query(input, executionOptions) {
const path = slPath(input.connectionId, input.sourceName);
if (input.delete) {
const deleted = await project.fileStore.deleteFile(
path,
LOCAL_AUTHOR,
LOCAL_AUTHOR_EMAIL,
`Remove semantic-layer source: ${input.sourceName}`,
);
return { success: Boolean(deleted), sourceName: input.sourceName };
}
const yaml =
input.yaml ?? YAML.stringify({ ...input.source, name: input.sourceName }, { indent: 2, lineWidth: 0, version: '1.1' });
parseYamlRecord(yaml);
await project.fileStore.writeFile(
path,
`${yaml.trimEnd()}\n`,
LOCAL_AUTHOR,
LOCAL_AUTHOR_EMAIL,
`Update semantic-layer source: ${input.sourceName}`,
);
return { success: true, sourceName: input.sourceName, yaml: `${yaml.trimEnd()}\n` };
},
async validate(input) {
if (options.semanticLayerCompute) {
const connectionId = assertSafeConnectionId(input.connectionId);
const result = await options.semanticLayerCompute.validateSources({
sources: await loadComputableSources(project, connectionId),
dialect: dialectForDriver(project.config.connections[connectionId]?.driver),
recentlyTouched: input.names,
});
return {
success: result.valid,
errors: result.errors,
warnings: result.warnings,
};
}
const names = new Set(input.names ?? []);
const paths = await listSlPaths(project, input.connectionId);
const errors: string[] = [];
for (const path of paths) {
const sourceName = sourceNameFromPath(path);
if (names.size > 0 && !names.has(sourceName)) {
continue;
}
try {
const raw = await project.fileStore.readFile(path);
errors.push(...validateSourceRecord(sourceName, parseYamlRecord(raw.content)));
} catch (error) {
errors.push(`${sourceName}: ${error instanceof Error ? error.message : String(error)}`);
}
}
return {
success: errors.length === 0,
errors,
warnings: [SL_SHAPE_WARNING],
};
},
async query(input) {
if (!options.semanticLayerCompute) { if (!options.semanticLayerCompute) {
throw new Error( throw new Error('sl_query requires a semantic-layer query adapter.');
'sl_query requires a semantic-layer query adapter. Local stdio MCP exposes file-backed SL CRUD only.',
);
} }
return compileLocalSlQuery(project, { return compileLocalSlQuery(project, {
connectionId: input.connectionId, connectionId: input.connectionId,
@ -629,6 +210,7 @@ export function createLocalProjectMcpContextPorts(
execute: Boolean(options.queryExecutor), execute: Boolean(options.queryExecutor),
maxRows: input.query.limit, maxRows: input.query.limit,
queryExecutor: options.queryExecutor, queryExecutor: options.queryExecutor,
onProgress: executionOptions?.onProgress,
}); });
}, },
}, },
@ -651,114 +233,8 @@ export function createLocalProjectMcpContextPorts(
if (options.sqlAnalysis && options.localScan?.createConnector) { if (options.sqlAnalysis && options.localScan?.createConnector) {
ports.sqlExecution = { ports.sqlExecution = {
async execute(input) { async execute(input, executionOptions) {
return executeValidatedReadOnlySql(project, options, input); return executeValidatedReadOnlySql(project, options, input, executionOptions?.onProgress);
},
};
}
if (options.localIngest) {
ports.ingest = {
async trigger(input) {
const sourceDir = localIngestSourceDir(input.config);
if (input.adapter === 'metabase' && !sourceDir) {
const result = await (options.localIngest?.runLocalMetabaseIngest ?? runLocalMetabaseIngest)({
project,
adapters: options.localIngest?.adapters ?? createDefaultLocalIngestAdapters(project),
metabaseConnectionId: input.connectionId,
trigger: input.trigger,
jobIdFactory: options.localIngest?.jobIdFactory,
pullConfigOptions: options.localIngest?.pullConfigOptions,
agentRunner: options.localIngest?.agentRunner,
llmRuntime: options.localIngest?.llmRuntime,
memoryModel: options.localIngest?.memoryModel,
semanticLayerCompute: options.localIngest?.semanticLayerCompute ?? options.semanticLayerCompute,
queryExecutor: options.localIngest?.queryExecutor ?? options.queryExecutor,
logger: options.localIngest?.logger,
});
return {
runId: `metabase-fanout:${result.metabaseConnectionId}`,
jobId: undefined,
reportId: undefined,
fanout: {
status: result.status,
children: result.children.map((child) => ({
runId: child.report.runId,
jobId: child.report.jobId,
reportId: child.report.id,
targetConnectionId: child.targetConnectionId,
metabaseDatabaseId: child.metabaseDatabaseId,
})),
},
};
}
const executeLocalIngest = options.localIngest?.runLocalIngest ?? runLocalIngest;
const result = await executeLocalIngest({
project,
adapters: options.localIngest?.adapters ?? createDefaultLocalIngestAdapters(project),
adapter: input.adapter,
connectionId: input.connectionId,
sourceDir,
pullConfigOptions: options.localIngest?.pullConfigOptions,
trigger: input.trigger,
jobId: options.localIngest?.jobIdFactory?.(),
agentRunner: options.localIngest?.agentRunner,
llmRuntime: options.localIngest?.llmRuntime,
memoryModel: options.localIngest?.memoryModel,
semanticLayerCompute: options.localIngest?.semanticLayerCompute ?? options.semanticLayerCompute,
queryExecutor: options.localIngest?.queryExecutor ?? options.queryExecutor,
logger: options.localIngest?.logger,
});
return {
runId: result.report.runId,
jobId: result.report.jobId,
reportId: result.report.id,
};
},
async status(input) {
const report = await getLocalIngestStatus(project, input.runId);
return report ? statusFromIngestReport(report) : null;
},
async report(input) {
return getLocalIngestStatus(project, input.runId);
},
async replay(input) {
const report = await getLocalIngestStatus(project, input.runId);
return report ? ingestReportToMemoryFlowReplay(report) : null;
},
};
}
if (options.localScan) {
ports.scan = {
async trigger(input) {
return runLocalScan({
project,
connectionId: input.connectionId,
mode: input.mode,
detectRelationships: input.detectRelationships,
dryRun: input.dryRun,
trigger: 'mcp',
adapters: options.localScan?.adapters,
databaseIntrospectionUrl: options.localScan?.databaseIntrospectionUrl,
createConnector: options.localScan?.createConnector,
jobId: options.localScan?.jobIdFactory?.(),
now: options.localScan?.now,
});
},
async status(input) {
return getLocalScanStatus(project, input.runId);
},
async report(input) {
return getLocalScanReport(project, input.runId);
},
async listArtifacts(input) {
const report = await getLocalScanReport(project, input.runId);
return report ? listArtifactsForReport(project, input.runId, report) : null;
},
async readArtifact(input) {
return readScanArtifact(project, input.runId, input.path);
}, },
}; };
} }

File diff suppressed because it is too large Load diff

View file

@ -1,71 +1,8 @@
import { randomUUID } from 'node:crypto';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'; import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod'; import { registerKtxContextTools } from './context-tools.js';
import type { MemoryAgentInput } from '../memory/index.js'; import type { KtxMcpServerDeps, KtxMcpServerLike } from './types.js';
import { jsonErrorToolResult, jsonToolResult, registerKtxContextTools } from './context-tools.js';
import type { KtxMcpServerDeps, KtxMcpServerLike, MemoryCapturePort } from './types.js';
const memoryCaptureInputSchema = {
userMessage: z.string().min(1).describe('The user message that may contain durable knowledge.'),
assistantMessage: z.string().optional().describe('The assistant response that concluded the exchange.'),
connectionId: z.string().min(1).optional().describe('Optional connection id for semantic-layer capture.'),
};
const memoryCaptureStatusInputSchema = {
runId: z.string().min(1).describe('The memory capture run id returned by memory_capture.'),
};
function registerMemoryCaptureTools(deps: {
server: KtxMcpServerLike;
memoryCapture: MemoryCapturePort;
userContext: KtxMcpServerDeps['userContext'];
}): void {
deps.server.registerTool(
'memory_capture',
{
title: 'Memory Capture',
description:
'Capture durable knowledge and semantic-layer updates from the final user/assistant exchange. Returns a run id for polling.',
inputSchema: memoryCaptureInputSchema,
},
async (input) => {
const captureInput: MemoryAgentInput = {
userId: deps.userContext.userId,
chatId: `mcp-${randomUUID()}`,
userMessage: String(input.userMessage),
assistantMessage: typeof input.assistantMessage === 'string' ? input.assistantMessage : undefined,
connectionId: typeof input.connectionId === 'string' ? input.connectionId : undefined,
sourceType: 'external_ingest',
};
const result = await deps.memoryCapture.capture(captureInput);
return jsonToolResult(result);
},
);
deps.server.registerTool(
'memory_capture_status',
{
title: 'Memory Capture Status',
description: 'Read the current or final status for a memory capture run.',
inputSchema: memoryCaptureStatusInputSchema,
},
async (input) => {
const runId = String(input.runId);
const status = await deps.memoryCapture.status(runId);
return status ? jsonToolResult(status) : jsonErrorToolResult(`Memory capture run "${runId}" was not found.`);
},
);
}
export function createKtxMcpServer(deps: KtxMcpServerDeps): KtxMcpServerDeps['server'] { export function createKtxMcpServer(deps: KtxMcpServerDeps): KtxMcpServerDeps['server'] {
if (deps.memoryCapture) {
registerMemoryCaptureTools({
server: deps.server,
memoryCapture: deps.memoryCapture,
userContext: deps.userContext,
});
}
if (deps.contextTools) { if (deps.contextTools) {
registerKtxContextTools({ registerKtxContextTools({
server: deps.server, server: deps.server,
@ -86,7 +23,6 @@ export function createDefaultKtxMcpServer(
}); });
createKtxMcpServer({ createKtxMcpServer({
server: server as KtxMcpServerLike, server: server as KtxMcpServerLike,
memoryCapture: deps.memoryCapture,
userContext: deps.userContext, userContext: deps.userContext,
contextTools: deps.contextTools, contextTools: deps.contextTools,
}); });

View file

@ -1,16 +1,7 @@
import type { IngestReportSnapshot, MemoryFlowReplayInput, TableUsageOutput } from '../ingest/index.js'; import type { MemoryIngestService } from '../memory/index.js';
import type { MemoryCaptureService } from '../memory/index.js';
import type { KtxEntityDetailsInput, KtxEntityDetailsResponse } from '../scan/entity-details.js'; import type { KtxEntityDetailsInput, KtxEntityDetailsResponse } from '../scan/entity-details.js';
import type { KtxScanMode, KtxScanReport } from '../scan/index.js';
import type { KtxDiscoverDataInput, KtxDiscoverDataResponse } from '../search/index.js'; import type { KtxDiscoverDataInput, KtxDiscoverDataResponse } from '../search/index.js';
import type { import type { KtxDictionarySearchInput, KtxDictionarySearchResponse, SemanticLayerQueryInput } from '../sl/index.js';
KtxDictionarySearchInput,
KtxDictionarySearchResponse,
SemanticLayerQueryInput,
SlDictionaryMatch,
SlSearchLaneSummary,
SlSearchMatchReason,
} from '../sl/index.js';
import type { WikiSearchLaneSummary, WikiSearchMatchReason } from '../wiki/index.js'; import type { WikiSearchLaneSummary, WikiSearchMatchReason } from '../wiki/index.js';
export interface KtxMcpTextContent { export interface KtxMcpTextContent {
@ -18,15 +9,38 @@ export interface KtxMcpTextContent {
text: string; text: string;
} }
export interface KtxMcpToolResult<T extends object = object> { export type NonArrayObject = object & { length?: never };
export interface KtxMcpToolResult<T extends NonArrayObject = NonArrayObject> {
content: KtxMcpTextContent[]; content: KtxMcpTextContent[];
structuredContent?: T; structuredContent?: T;
isError?: true; isError?: true;
} }
export interface MemoryCapturePort { interface KtxMcpProgressEvent {
capture: MemoryCaptureService['capture']; progress: number;
status: MemoryCaptureService['status']; total?: number;
message: string;
}
export type KtxMcpProgressCallback = (event: KtxMcpProgressEvent) => void | Promise<void>;
export interface KtxMcpToolHandlerContext {
_meta?: { progressToken?: string | number; [key: string]: unknown };
sendNotification?: (notification: {
method: 'notifications/progress';
params: {
progressToken: string | number;
progress: number;
total?: number;
message?: string;
};
}) => Promise<void>;
}
export interface MemoryIngestPort {
ingest: MemoryIngestService['ingest'];
status: MemoryIngestService['status'];
} }
export interface KtxMcpUserContext { export interface KtxMcpUserContext {
@ -40,8 +54,10 @@ export interface KtxMcpServerLike {
title?: string; title?: string;
description?: string; description?: string;
inputSchema: unknown; inputSchema: unknown;
outputSchema?: unknown;
annotations?: Record<string, unknown>;
}, },
handler: (input: Record<string, unknown>) => Promise<unknown>, handler: (input: Record<string, unknown>, context?: KtxMcpToolHandlerContext) => Promise<unknown>,
): void; ): void;
} }
@ -51,18 +67,8 @@ export interface KtxConnectionSummary {
connectionType: string; connectionType: string;
} }
export interface KtxConnectionTestResponse {
id: string;
connectionType: string;
ok: boolean;
tableCount: number | null;
message: string;
warnings: string[];
}
export interface KtxConnectionsMcpPort { export interface KtxConnectionsMcpPort {
list(): Promise<KtxConnectionSummary[]>; list(): Promise<KtxConnectionSummary[]>;
test?(input: { connectionId: string }): Promise<KtxConnectionTestResponse | null>;
} }
export interface KtxKnowledgeSearchResult { export interface KtxKnowledgeSearchResult {
@ -90,62 +96,9 @@ export interface KtxKnowledgePage {
slRefs?: string[]; slRefs?: string[];
} }
interface KtxHistoricSqlKnowledgeUsage {
executions: number;
distinct_users: number;
first_seen: string;
last_seen: string;
p50_runtime_ms: number | null;
p95_runtime_ms: number | null;
error_rate: number;
rows_produced?: number;
}
export interface KtxKnowledgeWriteResponse {
success: boolean;
key: string;
action: 'created' | 'updated';
}
export interface KtxKnowledgeMcpPort { export interface KtxKnowledgeMcpPort {
search(input: { userId: string; query: string; limit: number }): Promise<KtxKnowledgeSearchResponse>; search(input: { userId: string; query: string; limit: number }): Promise<KtxKnowledgeSearchResponse>;
read(input: { userId: string; key: string }): Promise<KtxKnowledgePage | null>; read(input: { userId: string; key: string }): Promise<KtxKnowledgePage | null>;
write(input: {
userId: string;
key: string;
summary: string;
content: string;
tags?: string[];
refs?: string[];
slRefs?: string[];
source?: string;
intent?: string;
tables?: string[];
representativeSql?: string;
usage?: KtxHistoricSqlKnowledgeUsage;
fingerprints?: string[];
}): Promise<KtxKnowledgeWriteResponse>;
}
export interface KtxSemanticLayerSourceSummary {
connectionId: string;
connectionName: string;
name: string;
description?: string;
columnCount: number;
measureCount: number;
joinCount: number;
frequencyTier?: TableUsageOutput['frequencyTier'];
snippet?: string;
score?: number;
matchReasons?: SlSearchMatchReason[];
dictionaryMatches?: SlDictionaryMatch[];
lanes?: SlSearchLaneSummary[];
}
export interface KtxSemanticLayerListResponse {
sources: KtxSemanticLayerSourceSummary[];
totalSources: number;
} }
export interface KtxSemanticLayerReadResponse { export interface KtxSemanticLayerReadResponse {
@ -153,21 +106,6 @@ export interface KtxSemanticLayerReadResponse {
yaml: string; yaml: string;
} }
export interface KtxSemanticLayerWriteResponse {
success: boolean;
sourceName: string;
yaml?: string;
errors?: string[];
warnings?: string[];
commitHash?: string;
}
export interface KtxSemanticLayerValidationResponse {
success: boolean;
errors: string[];
warnings: string[];
}
export interface KtxSemanticLayerQueryResponse { export interface KtxSemanticLayerQueryResponse {
sql: string; sql: string;
headers: string[]; headers: string[];
@ -177,143 +115,11 @@ export interface KtxSemanticLayerQueryResponse {
} }
export interface KtxSemanticLayerMcpPort { export interface KtxSemanticLayerMcpPort {
listSources(input: { connectionId?: string; query?: string }): Promise<KtxSemanticLayerListResponse>;
readSource(input: { connectionId: string; sourceName: string }): Promise<KtxSemanticLayerReadResponse | null>; readSource(input: { connectionId: string; sourceName: string }): Promise<KtxSemanticLayerReadResponse | null>;
writeSource(input: { query(
connectionId: string; input: { connectionId?: string; query: SemanticLayerQueryInput },
sourceName: string; options?: { onProgress?: KtxMcpProgressCallback },
yaml?: string; ): Promise<KtxSemanticLayerQueryResponse>;
source?: Record<string, unknown>;
delete?: boolean;
}): Promise<KtxSemanticLayerWriteResponse>;
validate(input: { connectionId: string; names?: string[] }): Promise<KtxSemanticLayerValidationResponse>;
query(input: { connectionId?: string; query: SemanticLayerQueryInput }): Promise<KtxSemanticLayerQueryResponse>;
}
export type KtxIngestTriggerKind = 'upload' | 'scheduled_pull' | 'manual_resync';
interface KtxIngestTriggerFanoutChild {
runId: string;
jobId: string;
reportId: string;
targetConnectionId: string;
metabaseDatabaseId: number;
}
export interface KtxIngestTriggerResponse {
runId: string;
jobId?: string;
reportId?: string;
fanout?: {
status: 'all_succeeded' | 'partial_failure' | 'all_failed';
children: KtxIngestTriggerFanoutChild[];
};
}
export interface KtxIngestDiffSummary {
added: number;
modified: number;
deleted: number;
unchanged: number;
}
export interface KtxIngestWorkUnitSummary {
unitKey: string;
rawFiles: string[];
peerFileIndex: string[];
dependencyPaths: string[];
}
export interface KtxIngestStatusResponse {
runId: string;
jobId?: string;
reportId?: string;
status: string;
stage?: string;
progress?: number;
errors?: string[];
done: boolean;
adapter?: string;
connectionId?: string;
sourceDir?: string | null;
syncId?: string;
startedAt?: string;
completedAt?: string;
previousRunId?: string | null;
diffSummary?: KtxIngestDiffSummary;
workUnitCount?: number;
rawFileCount?: number;
workUnits?: KtxIngestWorkUnitSummary[];
evictionDeletedRawPaths?: string[];
}
export interface KtxIngestMcpPort {
trigger(input: {
adapter: string;
connectionId: string;
config?: unknown;
trigger: KtxIngestTriggerKind;
}): Promise<KtxIngestTriggerResponse>;
status(input: { runId: string }): Promise<KtxIngestStatusResponse | null>;
report?(input: { runId: string }): Promise<IngestReportSnapshot | null>;
replay?(input: { runId: string }): Promise<MemoryFlowReplayInput | null>;
}
interface KtxScanTriggerResponse {
runId: string;
status: 'done';
done: true;
connectionId: string;
mode: KtxScanMode;
dryRun: boolean;
syncId: string;
report: KtxScanReport;
}
interface KtxScanStatusResponse {
runId: string;
status: string;
done: boolean;
connectionId: string;
mode: KtxScanMode;
dryRun: boolean;
syncId: string;
progress: number;
startedAt: string;
completedAt: string;
reportPath: string | null;
warnings: KtxScanReport['warnings'];
}
export type KtxScanArtifactType = 'report' | 'raw_source' | 'manifest_shard' | 'enrichment_artifact';
export interface KtxScanArtifactSummary {
path: string;
type: KtxScanArtifactType;
size?: number;
}
export interface KtxScanArtifactListResponse {
runId: string;
artifacts: KtxScanArtifactSummary[];
}
export interface KtxScanArtifactReadResponse extends KtxScanArtifactSummary {
runId: string;
content: string;
}
export interface KtxScanMcpPort {
trigger(input: {
connectionId: string;
mode?: KtxScanMode;
detectRelationships: boolean;
dryRun: boolean;
}): Promise<KtxScanTriggerResponse>;
status(input: { runId: string }): Promise<KtxScanStatusResponse | null>;
report(input: { runId: string }): Promise<KtxScanReport | null>;
listArtifacts?(input: { runId: string }): Promise<KtxScanArtifactListResponse | null>;
readArtifact?(input: { runId: string; path: string }): Promise<KtxScanArtifactReadResponse | null>;
} }
export interface KtxEntityDetailsMcpPort { export interface KtxEntityDetailsMcpPort {
@ -336,7 +142,10 @@ export interface KtxSqlExecutionResponse {
} }
export interface KtxSqlExecutionMcpPort { export interface KtxSqlExecutionMcpPort {
execute(input: { connectionId: string; sql: string; maxRows: number }): Promise<KtxSqlExecutionResponse>; execute(
input: { connectionId: string; sql: string; maxRows: number },
options?: { onProgress?: KtxMcpProgressCallback },
): Promise<KtxSqlExecutionResponse>;
} }
export interface KtxMcpContextPorts { export interface KtxMcpContextPorts {
@ -347,13 +156,11 @@ export interface KtxMcpContextPorts {
dictionarySearch?: KtxDictionarySearchMcpPort; dictionarySearch?: KtxDictionarySearchMcpPort;
discover?: KtxDiscoverDataMcpPort; discover?: KtxDiscoverDataMcpPort;
sqlExecution?: KtxSqlExecutionMcpPort; sqlExecution?: KtxSqlExecutionMcpPort;
ingest?: KtxIngestMcpPort; memoryIngest?: MemoryIngestPort;
scan?: KtxScanMcpPort;
} }
export interface KtxMcpServerDeps { export interface KtxMcpServerDeps {
server: KtxMcpServerLike; server: KtxMcpServerLike;
memoryCapture?: MemoryCapturePort;
userContext: KtxMcpUserContext; userContext: KtxMcpUserContext;
contextTools?: KtxMcpContextPorts; contextTools?: KtxMcpContextPorts;
} }

View file

@ -8,13 +8,13 @@ export {
stepBudgetFor, stepBudgetFor,
} from './capture-signals.js'; } from './capture-signals.js';
export { MemoryAgentService } from './memory-agent.service.js'; export { MemoryAgentService } from './memory-agent.service.js';
export { createLocalProjectMemoryCapture, type CreateLocalProjectMemoryCaptureOptions } from './local-memory.js'; export { createLocalProjectMemoryIngest, type CreateLocalProjectMemoryIngestOptions } from './local-memory.js';
export { LocalMemoryRunStore, type LocalMemoryRunStoreOptions } from './local-memory-runs.js'; export { LocalMemoryRunStore, type LocalMemoryRunStoreOptions } from './local-memory-runs.js';
export { export {
MemoryCaptureService, MemoryIngestService,
type MemoryCaptureServiceDeps, type MemoryIngestServiceDeps,
type MemoryCaptureStartResult, type MemoryIngestStartResult,
type MemoryCaptureStatus, type MemoryIngestStatus,
type MemoryRunRecord, type MemoryRunRecord,
type MemoryRunStatus, type MemoryRunStatus,
type MemoryRunStorePort, type MemoryRunStorePort,

View file

@ -3,7 +3,7 @@ import { tmpdir } from 'node:os';
import { join } from 'node:path'; import { join } from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { initKtxProject } from '../project/index.js'; import { initKtxProject } from '../project/index.js';
import { createLocalProjectMemoryCapture } from './local-memory.js'; import { createLocalProjectMemoryIngest } from './local-memory.js';
import { LocalMemoryRunStore } from './local-memory-runs.js'; import { LocalMemoryRunStore } from './local-memory-runs.js';
vi.mock('ai', () => ({ vi.mock('ai', () => ({
@ -77,7 +77,7 @@ describe('LocalMemoryRunStore', () => {
}); });
}); });
describe('createLocalProjectMemoryCapture', () => { describe('createLocalProjectMemoryIngest', () => {
let tempDir: string; let tempDir: string;
beforeEach(async () => { beforeEach(async () => {
@ -110,13 +110,13 @@ describe('createLocalProjectMemoryCapture', () => {
}, },
}; };
const capture = createLocalProjectMemoryCapture(project, { const ingest = createLocalProjectMemoryIngest(project, {
agentRunner: agentRunner as never, agentRunner: agentRunner as never,
runIdFactory: () => 'memory-run-1', runIdFactory: () => 'memory-run-1',
}); });
await expect( await expect(
capture.capture({ ingest.ingest({
userId: 'local-user', userId: 'local-user',
chatId: 'chat-1', chatId: 'chat-1',
userMessage: 'define revenue as paid order value net of refunds', userMessage: 'define revenue as paid order value net of refunds',
@ -124,12 +124,12 @@ describe('createLocalProjectMemoryCapture', () => {
sourceType: 'external_ingest', sourceType: 'external_ingest',
}), }),
).resolves.toEqual({ runId: 'memory-run-1' }); ).resolves.toEqual({ runId: 'memory-run-1' });
await capture.waitForRun('memory-run-1'); await ingest.waitForRun('memory-run-1');
await expect(access(join(project.projectDir, '.ktx/db.sqlite'))).resolves.toBeUndefined(); await expect(access(join(project.projectDir, '.ktx/db.sqlite'))).resolves.toBeUndefined();
await expectPathMissing(join(project.projectDir, '.ktx/memory-runs/memory-run-1.json')); await expectPathMissing(join(project.projectDir, '.ktx/memory-runs/memory-run-1.json'));
await expect(capture.status('memory-run-1')).resolves.toMatchObject({ await expect(ingest.status('memory-run-1')).resolves.toMatchObject({
runId: 'memory-run-1', runId: 'memory-run-1',
status: 'done', status: 'done',
done: true, done: true,
@ -172,12 +172,12 @@ describe('createLocalProjectMemoryCapture', () => {
}, },
}; };
const capture = createLocalProjectMemoryCapture(project, { const ingest = createLocalProjectMemoryIngest(project, {
agentRunner: agentRunner as never, agentRunner: agentRunner as never,
runIdFactory: () => 'memory-run-2', runIdFactory: () => 'memory-run-2',
}); });
await capture.capture({ await ingest.ingest({
userId: 'local-user', userId: 'local-user',
chatId: 'chat-2', chatId: 'chat-2',
userMessage: 'going forward define orders count as count of public orders', userMessage: 'going forward define orders count as count of public orders',
@ -185,12 +185,12 @@ describe('createLocalProjectMemoryCapture', () => {
connectionId: 'warehouse', connectionId: 'warehouse',
sourceType: 'external_ingest', sourceType: 'external_ingest',
}); });
await capture.waitForRun('memory-run-2'); await ingest.waitForRun('memory-run-2');
await expect(access(join(project.projectDir, '.ktx/db.sqlite'))).resolves.toBeUndefined(); await expect(access(join(project.projectDir, '.ktx/db.sqlite'))).resolves.toBeUndefined();
await expectPathMissing(join(project.projectDir, '.ktx/memory-runs/memory-run-2.json')); await expectPathMissing(join(project.projectDir, '.ktx/memory-runs/memory-run-2.json'));
await expect(capture.status('memory-run-2')).resolves.toMatchObject({ await expect(ingest.status('memory-run-2')).resolves.toMatchObject({
runId: 'memory-run-2', runId: 'memory-run-2',
status: 'done', status: 'done',
captured: { wiki: [], sl: ['orders'], xrefs: [] }, captured: { wiki: [], sl: ['orders'], xrefs: [] },

View file

@ -51,7 +51,7 @@ import {
} from '../wiki/index.js'; } from '../wiki/index.js';
import { LocalMemoryRunStore } from './local-memory-runs.js'; import { LocalMemoryRunStore } from './local-memory-runs.js';
import { MemoryAgentService } from './memory-agent.service.js'; import { MemoryAgentService } from './memory-agent.service.js';
import { MemoryCaptureService } from './memory-runs.js'; import { MemoryIngestService } from './memory-runs.js';
import type { import type {
MemoryConnectionPort, MemoryConnectionPort,
MemoryFileStorePort, MemoryFileStorePort,
@ -64,9 +64,9 @@ import type {
const promptsDir = fileURLToPath(new URL('../../prompts', import.meta.url)); const promptsDir = fileURLToPath(new URL('../../prompts', import.meta.url));
const skillsDir = fileURLToPath(new URL('../../skills', import.meta.url)); const skillsDir = fileURLToPath(new URL('../../skills', import.meta.url));
const LOCAL_AUTHOR = { name: 'KTX Local', email: 'local@ktx.local' }; const LOCAL_AUTHOR = { name: 'KTX Local', email: 'local@ktx.local' };
const LOCAL_SHAPE_WARNING = 'Local memory capture validates semantic-layer YAML shape only.'; const LOCAL_SHAPE_WARNING = 'Local memory ingest validates semantic-layer YAML shape only.';
export interface CreateLocalProjectMemoryCaptureOptions { export interface CreateLocalProjectMemoryIngestOptions {
llmRuntime?: KtxLlmRuntimePort; llmRuntime?: KtxLlmRuntimePort;
agentRunner?: AgentRunnerPort; agentRunner?: AgentRunnerPort;
memoryModel?: string; memoryModel?: string;
@ -76,10 +76,10 @@ export interface CreateLocalProjectMemoryCaptureOptions {
logger?: KtxLogger; logger?: KtxLogger;
} }
export function createLocalProjectMemoryCapture( export function createLocalProjectMemoryIngest(
project: KtxLocalProject, project: KtxLocalProject,
options: CreateLocalProjectMemoryCaptureOptions = {}, options: CreateLocalProjectMemoryIngestOptions = {},
): MemoryCaptureService { ): MemoryIngestService {
const logger = options.logger ?? noopLogger; const logger = options.logger ?? noopLogger;
const rootFileStore = new LocalMemoryFileStore(project.fileStore); const rootFileStore = new LocalMemoryFileStore(project.fileStore);
const embedding = new NoopEmbeddingPort(); const embedding = new NoopEmbeddingPort();
@ -139,7 +139,7 @@ export function createLocalProjectMemoryCapture(
toolsetFactory, toolsetFactory,
logger, logger,
}); });
return new MemoryCaptureService({ return new MemoryIngestService({
memoryAgent, memoryAgent,
runs: new LocalMemoryRunStore({ projectDir: project.projectDir, idFactory: options.runIdFactory }), runs: new LocalMemoryRunStore({ projectDir: project.projectDir, idFactory: options.runIdFactory }),
}); });
@ -147,7 +147,7 @@ export function createLocalProjectMemoryCapture(
function requireLlmRuntime(runtime: KtxLlmRuntimePort | null | undefined): KtxLlmRuntimePort { function requireLlmRuntime(runtime: KtxLlmRuntimePort | null | undefined): KtxLlmRuntimePort {
if (!runtime) { if (!runtime) {
throw new Error('createLocalProjectMemoryCapture requires llm.provider.backend or an injected agentRunner'); throw new Error('createLocalProjectMemoryIngest requires llm.provider.backend or an injected agentRunner');
} }
return runtime; return runtime;
} }

View file

@ -1,6 +1,6 @@
import { describe, expect, it, vi } from 'vitest'; import { describe, expect, it, vi } from 'vitest';
import type { MemoryAgentInput, MemoryAgentResult, MemoryAgentService } from './index.js'; import type { MemoryAgentInput, MemoryAgentResult, MemoryAgentService } from './index.js';
import { MemoryCaptureService, type MemoryRunStorePort } from './memory-runs.js'; import { MemoryIngestService, type MemoryRunStorePort } from './memory-runs.js';
class InMemoryRunStore implements MemoryRunStorePort { class InMemoryRunStore implements MemoryRunStorePort {
readonly rows = new Map< readonly rows = new Map<
@ -74,32 +74,32 @@ function deferred<T>() {
} }
function buildService(): { function buildService(): {
capture: MemoryCaptureService; ingest: MemoryIngestService;
store: InMemoryRunStore; store: InMemoryRunStore;
ingest: ReturnType<typeof vi.fn>; memoryAgentIngest: ReturnType<typeof vi.fn>;
run: ReturnType<typeof deferred<MemoryAgentResult>>; run: ReturnType<typeof deferred<MemoryAgentResult>>;
} { } {
const store = new InMemoryRunStore(); const store = new InMemoryRunStore();
const run = deferred<MemoryAgentResult>(); const run = deferred<MemoryAgentResult>();
const ingest = vi.fn<MemoryAgentService['ingest']>().mockReturnValue(run.promise); const memoryAgentIngest = vi.fn<MemoryAgentService['ingest']>().mockReturnValue(run.promise);
const memoryAgent = { ingest }; const memoryAgent = { ingest: memoryAgentIngest };
return { return {
capture: new MemoryCaptureService({ memoryAgent, runs: store }), ingest: new MemoryIngestService({ memoryAgent, runs: store }),
store, store,
ingest, memoryAgentIngest,
run, run,
}; };
} }
describe('MemoryCaptureService', () => { describe('MemoryIngestService', () => {
it('creates a run, executes memory capture, and stores a done summary', async () => { it('creates a run, executes memory ingest, and stores a done summary', async () => {
const result: MemoryAgentResult = { const result: MemoryAgentResult = {
signalDetected: true, signalDetected: true,
actions: [{ target: 'wiki', type: 'created', key: 'revenue', detail: 'captured revenue definition' }], actions: [{ target: 'wiki', type: 'created', key: 'revenue', detail: 'captured revenue definition' }],
skillsLoaded: ['wiki_capture'], skillsLoaded: ['wiki_capture'],
commitHash: 'abc123', commitHash: 'abc123',
}; };
const { capture, store, ingest, run } = buildService(); const { ingest, store, memoryAgentIngest, run } = buildService();
const input: MemoryAgentInput = { const input: MemoryAgentInput = {
userId: 'user-1', userId: 'user-1',
@ -109,21 +109,21 @@ describe('MemoryCaptureService', () => {
connectionId: '00000000-0000-0000-0000-000000000001', connectionId: '00000000-0000-0000-0000-000000000001',
}; };
const started = await capture.capture(input); const started = await ingest.ingest(input);
expect(started.runId).toBe('run-1'); expect(started.runId).toBe('run-1');
expect(ingest).toHaveBeenCalledWith(input); expect(memoryAgentIngest).toHaveBeenCalledWith(input);
await expect(capture.status(started.runId)).resolves.toMatchObject({ await expect(ingest.status(started.runId)).resolves.toMatchObject({
runId: 'run-1', runId: 'run-1',
status: 'running', status: 'running',
stage: 'capturing', stage: 'ingesting',
done: false, done: false,
}); });
run.resolve(result); run.resolve(result);
await capture.waitForRun(started.runId); await ingest.waitForRun(started.runId);
const status = await capture.status(started.runId); const status = await ingest.status(started.runId);
expect(status).toEqual({ expect(status).toEqual({
runId: 'run-1', runId: 'run-1',
stage: 'done', stage: 'done',
@ -142,10 +142,10 @@ describe('MemoryCaptureService', () => {
expect(store.rows.get('run-1')?.inputHash).toHaveLength(64); expect(store.rows.get('run-1')?.inputHash).toHaveLength(64);
}); });
it('stores no-signal captures as done with empty captured arrays', async () => { it('stores no-signal ingests as done with empty captured arrays', async () => {
const { capture, run } = buildService(); const { ingest, run } = buildService();
const started = await capture.capture({ const started = await ingest.ingest({
userId: 'user-1', userId: 'user-1',
chatId: 'chat-2', chatId: 'chat-2',
userMessage: 'Thanks.', userMessage: 'Thanks.',
@ -157,9 +157,9 @@ describe('MemoryCaptureService', () => {
skillsLoaded: [], skillsLoaded: [],
commitHash: null, commitHash: null,
}); });
await capture.waitForRun(started.runId); await ingest.waitForRun(started.runId);
await expect(capture.status(started.runId)).resolves.toMatchObject({ await expect(ingest.status(started.runId)).resolves.toMatchObject({
done: true, done: true,
status: 'done', status: 'done',
captured: { wiki: [], sl: [], xrefs: [] }, captured: { wiki: [], sl: [], xrefs: [] },
@ -172,16 +172,16 @@ describe('MemoryCaptureService', () => {
const memoryAgent = { const memoryAgent = {
ingest: vi.fn<MemoryAgentService['ingest']>().mockRejectedValue(new Error('LLM provider missing')), ingest: vi.fn<MemoryAgentService['ingest']>().mockRejectedValue(new Error('LLM provider missing')),
}; };
const capture = new MemoryCaptureService({ memoryAgent, runs: store }); const ingest = new MemoryIngestService({ memoryAgent, runs: store });
const started = await capture.capture({ const started = await ingest.ingest({
userId: 'user-1', userId: 'user-1',
chatId: 'chat-3', chatId: 'chat-3',
userMessage: 'Remember this.', userMessage: 'Remember this.',
}); });
await capture.waitForRun(started.runId); await ingest.waitForRun(started.runId);
await expect(capture.status(started.runId)).resolves.toMatchObject({ await expect(ingest.status(started.runId)).resolves.toMatchObject({
done: true, done: true,
status: 'error', status: 'error',
stage: 'error', stage: 'error',
@ -191,8 +191,8 @@ describe('MemoryCaptureService', () => {
}); });
it('returns null for an unknown run id', async () => { it('returns null for an unknown run id', async () => {
const { capture } = buildService(); const { ingest } = buildService();
await expect(capture.status('missing')).resolves.toBeNull(); await expect(ingest.status('missing')).resolves.toBeNull();
}); });
}); });

View file

@ -21,16 +21,16 @@ export interface MemoryRunStorePort {
findById(id: string): Promise<MemoryRunRecord | null>; findById(id: string): Promise<MemoryRunRecord | null>;
} }
export interface MemoryCaptureServiceDeps { export interface MemoryIngestServiceDeps {
memoryAgent: Pick<MemoryAgentService, 'ingest'>; memoryAgent: Pick<MemoryAgentService, 'ingest'>;
runs: MemoryRunStorePort; runs: MemoryRunStorePort;
} }
export interface MemoryCaptureStartResult { export interface MemoryIngestStartResult {
runId: string; runId: string;
} }
export interface MemoryCaptureStatus { export interface MemoryIngestStatus {
runId: string; runId: string;
status: MemoryRunStatus; status: MemoryRunStatus;
stage: string; stage: string;
@ -55,7 +55,7 @@ function inputHash(input: MemoryAgentInput): string {
return createHash('sha256').update(stableInput).digest('hex'); return createHash('sha256').update(stableInput).digest('hex');
} }
function capturedKeys(actions: MemoryAction[]): MemoryCaptureStatus['captured'] { function capturedKeys(actions: MemoryAction[]): MemoryIngestStatus['captured'] {
const wiki = new Set<string>(); const wiki = new Set<string>();
const sl = new Set<string>(); const sl = new Set<string>();
const xrefs = new Set<string>(); const xrefs = new Set<string>();
@ -78,20 +78,20 @@ function capturedKeys(actions: MemoryAction[]): MemoryCaptureStatus['captured']
}; };
} }
export class MemoryCaptureService { export class MemoryIngestService {
private readonly inFlight = new Map<string, Promise<void>>(); private readonly inFlight = new Map<string, Promise<void>>();
constructor(private readonly deps: MemoryCaptureServiceDeps) {} constructor(private readonly deps: MemoryIngestServiceDeps) {}
async capture(input: MemoryAgentInput): Promise<MemoryCaptureStartResult> { async ingest(input: MemoryAgentInput): Promise<MemoryIngestStartResult> {
const row = await this.deps.runs.createRunning({ const row = await this.deps.runs.createRunning({
inputHash: inputHash(input), inputHash: inputHash(input),
chatId: input.chatId, chatId: input.chatId,
}); });
await this.deps.runs.markRunning(row.id, 'capturing'); await this.deps.runs.markRunning(row.id, 'ingesting');
const run = this.runCapture(row.id, input); const run = this.runIngest(row.id, input);
this.inFlight.set(row.id, run); this.inFlight.set(row.id, run);
run.finally(() => this.inFlight.delete(row.id)).catch(() => undefined); run.finally(() => this.inFlight.delete(row.id)).catch(() => undefined);
@ -102,7 +102,7 @@ export class MemoryCaptureService {
await this.inFlight.get(runId); await this.inFlight.get(runId);
} }
private async runCapture(runId: string, input: MemoryAgentInput): Promise<void> { private async runIngest(runId: string, input: MemoryAgentInput): Promise<void> {
try { try {
const outputSummary = await this.deps.memoryAgent.ingest(input); const outputSummary = await this.deps.memoryAgent.ingest(input);
await this.deps.runs.markDone(runId, outputSummary); await this.deps.runs.markDone(runId, outputSummary);
@ -111,7 +111,7 @@ export class MemoryCaptureService {
} }
} }
async status(runId: string): Promise<MemoryCaptureStatus | null> { async status(runId: string): Promise<MemoryIngestStatus | null> {
const row = await this.deps.runs.findById(runId); const row = await this.deps.runs.findById(runId);
if (!row) { if (!row) {
return null; return null;

View file

@ -182,6 +182,46 @@ grain: []
}); });
}); });
it('strips authoring-only fields (usage, inherits_columns_from) before sending sources to the daemon', async () => {
await project.fileStore.writeFile(
'semantic-layer/warehouse/_schema/public.yaml',
`tables:
invoices:
table: public.invoices
columns:
- name: invoice_id
type: number
pk: true
- name: amount
type: number
usage:
narrative: Activation policy windows table for invoice analytics.
frequencyTier: mid
commonFilters:
- amount
commonGroupBys: []
commonJoins: []
staleSince: null
`,
'ktx',
'ktx@example.com',
'Add manifest shard with usage',
);
await compileLocalSlQuery(project, {
connectionId: 'warehouse',
query: { measures: ['sum(invoices.amount)'], dimensions: [] },
compute,
});
const lastCall = (compute.query as ReturnType<typeof vi.fn>).mock.calls.at(-1)?.[0];
const invoices = lastCall?.sources.find((s: Record<string, unknown>) => s.name === 'invoices');
expect(invoices).toBeDefined();
expect(invoices).not.toHaveProperty('usage');
expect(invoices).not.toHaveProperty('inherits_columns_from');
expect(invoices).not.toHaveProperty('source_type');
});
it('resolves the only configured connection when connectionId is omitted', async () => { it('resolves the only configured connection when connectionId is omitted', async () => {
await compileLocalSlQuery(project, { await compileLocalSlQuery(project, {
query: { measures: ['orders.order_count'], dimensions: [] }, query: { measures: ['orders.order_count'], dimensions: [] },
@ -236,6 +276,43 @@ grain: []
}); });
}); });
it('emits progress while compiling and executing a local semantic-layer query', async () => {
const progress: Array<{ progress: number; message: string }> = [];
const queryExecutor = {
execute: vi.fn(async () => ({
headers: ['status', 'order_count'],
rows: [['paid', 2]],
totalRows: 1,
command: 'SELECT',
rowCount: 1,
})),
};
const result = await compileLocalSlQuery(project, {
connectionId: 'warehouse',
query: {
measures: ['orders.order_count'],
dimensions: ['orders.status'],
limit: 25,
},
compute,
execute: true,
maxRows: 10,
queryExecutor,
onProgress: (event) => {
progress.push({ progress: event.progress, message: event.message });
},
});
expect(result.totalRows).toBe(1);
expect(progress).toEqual([
{ progress: 0, message: 'Compiling query' },
{ progress: 0.3, message: 'Generating SQL' },
{ progress: 0.6, message: 'Executing' },
{ progress: 1, message: 'Fetched 1 rows' },
]);
});
it('requires a query executor for executed mode', async () => { it('requires a query executor for executed mode', async () => {
await expect( await expect(
compileLocalSlQuery(project, { compileLocalSlQuery(project, {

View file

@ -1,7 +1,9 @@
import type { KtxSqlQueryExecutorPort } from '../connections/index.js'; import type { KtxSqlQueryExecutorPort } from '../connections/index.js';
import type { KtxSemanticLayerComputePort } from '../daemon/index.js'; import type { KtxSemanticLayerComputePort } from '../daemon/index.js';
import type { KtxMcpProgressCallback } from '../mcp/types.js';
import type { KtxLocalProject } from '../project/index.js'; import type { KtxLocalProject } from '../project/index.js';
import { loadLocalSlSourceRecords } from './local-sl.js'; import { loadLocalSlSourceRecords } from './local-sl.js';
import { toResolvedWire } from './semantic-layer.service.js';
import type { SemanticLayerQueryExecutionResult, SemanticLayerQueryInput } from './types.js'; import type { SemanticLayerQueryExecutionResult, SemanticLayerQueryInput } from './types.js';
const COMPILE_ONLY_REASON = const COMPILE_ONLY_REASON =
@ -14,6 +16,7 @@ export interface CompileLocalSlQueryOptions {
execute?: boolean; execute?: boolean;
maxRows?: number; maxRows?: number;
queryExecutor?: KtxSqlQueryExecutorPort; queryExecutor?: KtxSqlQueryExecutorPort;
onProgress?: KtxMcpProgressCallback;
} }
export interface CompileLocalSlQueryResult extends SemanticLayerQueryExecutionResult { export interface CompileLocalSlQueryResult extends SemanticLayerQueryExecutionResult {
@ -75,10 +78,10 @@ function resolveLocalConnectionId(project: KtxLocalProject, requested: string |
async function loadComputableSources( async function loadComputableSources(
project: KtxLocalProject, project: KtxLocalProject,
connectionId: string, connectionId: string,
): Promise<Record<string, unknown>[]> { ): Promise<ReturnType<typeof toResolvedWire>[]> {
return (await loadLocalSlSourceRecords(project, { connectionId: assertSafeConnectionId(connectionId) })) return (await loadLocalSlSourceRecords(project, { connectionId: assertSafeConnectionId(connectionId) }))
.map((record) => ({ ...record.source })) .filter((record) => record.source.table || record.source.sql)
.filter((source) => source.table || source.sql); .map((record) => toResolvedWire(record.source));
} }
function headersFromColumns(columns: Array<Record<string, unknown>>): string[] { function headersFromColumns(columns: Array<Record<string, unknown>>): string[] {
@ -91,15 +94,20 @@ export async function compileLocalSlQuery(
project: KtxLocalProject, project: KtxLocalProject,
options: CompileLocalSlQueryOptions, options: CompileLocalSlQueryOptions,
): Promise<CompileLocalSlQueryResult> { ): Promise<CompileLocalSlQueryResult> {
await options.onProgress?.({ progress: 0, message: 'Compiling query' });
const connectionId = resolveLocalConnectionId(project, options.connectionId); const connectionId = resolveLocalConnectionId(project, options.connectionId);
const dialect = dialectForDriver(project.config.connections[connectionId]?.driver); const dialect = dialectForDriver(project.config.connections[connectionId]?.driver);
const sources = await loadComputableSources(project, connectionId);
await options.onProgress?.({ progress: 0.3, message: 'Generating SQL' });
const response = await options.compute.query({ const response = await options.compute.query({
sources: await loadComputableSources(project, connectionId), sources,
dialect, dialect,
query: options.query, query: options.query,
}); });
if (!options.execute) { if (!options.execute) {
await options.onProgress?.({ progress: 1, message: 'Fetched 0 rows' });
return { return {
connectionId, connectionId,
dialect: response.dialect, dialect: response.dialect,
@ -122,6 +130,7 @@ export async function compileLocalSlQuery(
} }
const maxRows = options.maxRows ?? options.query.limit; const maxRows = options.maxRows ?? options.query.limit;
await options.onProgress?.({ progress: 0.6, message: 'Executing' });
const execution = await options.queryExecutor.execute({ const execution = await options.queryExecutor.execute({
connectionId, connectionId,
projectDir: project.projectDir, projectDir: project.projectDir,
@ -129,6 +138,7 @@ export async function compileLocalSlQuery(
sql: response.sql, sql: response.sql,
maxRows, maxRows,
}); });
await options.onProgress?.({ progress: 1, message: `Fetched ${execution.totalRows} rows` });
return { return {
connectionId, connectionId,

8
pnpm-lock.yaml generated
View file

@ -136,6 +136,9 @@ importers:
commander: commander:
specifier: 14.0.3 specifier: 14.0.3
version: 14.0.3 version: 14.0.3
fflate:
specifier: ^0.8.2
version: 0.8.2
ink: ink:
specifier: ^7.0.2 specifier: ^7.0.2
version: 7.0.2(@types/react@19.2.14)(react@19.2.6) version: 7.0.2(@types/react@19.2.14)(react@19.2.6)
@ -3631,6 +3634,9 @@ packages:
resolution: {integrity: sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==} resolution: {integrity: sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==}
engines: {node: ^12.20 || >= 14.13} engines: {node: ^12.20 || >= 14.13}
fflate@0.8.2:
resolution: {integrity: sha512-cPJU47OaAoCbg0pBvzsgpTPhmhqI5eJjh/JIu8tPj5q+T7iLvW/JAYUqmE7KOB4R1ZyEhzBaIQpQpardBF5z8A==}
figures@2.0.0: figures@2.0.0:
resolution: {integrity: sha512-Oa2M9atig69ZkfwiApY8F2Yy+tzMbazyvqv21R0NsSC8floSOC09BbT1ITWAdoMGQvJ/aZnR1KMwdx9tvHnTNA==} resolution: {integrity: sha512-Oa2M9atig69ZkfwiApY8F2Yy+tzMbazyvqv21R0NsSC8floSOC09BbT1ITWAdoMGQvJ/aZnR1KMwdx9tvHnTNA==}
engines: {node: '>=4'} engines: {node: '>=4'}
@ -9983,6 +9989,8 @@ snapshots:
node-domexception: 1.0.0 node-domexception: 1.0.0
web-streams-polyfill: 3.3.3 web-streams-polyfill: 3.3.3
fflate@0.8.2: {}
figures@2.0.0: figures@2.0.0:
dependencies: dependencies:
escape-string-regexp: 1.0.5 escape-string-regexp: 1.0.5

View file

@ -261,6 +261,7 @@ describe('standalone example docs', () => {
assert.match(contextAsCode, /ktx ingest --all --no-input/); assert.match(contextAsCode, /ktx ingest --all --no-input/);
assert.match(quickstart, /schema context/); assert.match(quickstart, /schema context/);
assert.match(primarySources, /context:\n queryHistory:/); assert.match(primarySources, /context:\n queryHistory:/);
assert.match(rootReadme, /`ktx ingest <id>` \| Build context for one connection/);
assert.match(quickstart, /Databases:\n warehouse: deep context complete/); assert.match(quickstart, /Databases:\n warehouse: deep context complete/);
assert.match(quickstart, /Databases configured: yes \(warehouse\)/); assert.match(quickstart, /Databases configured: yes \(warehouse\)/);
assert.match(setupReference, /Databases configured: yes \(postgres-warehouse\)/); assert.match(setupReference, /Databases configured: yes \(postgres-warehouse\)/);