mirror of
https://github.com/Kaelio/ktx.git
synced 2026-07-13 11:22:11 +02:00
docs: expand claude-code spec to full llm parity
This commit is contained in:
parent
1899f763f6
commit
f7fbacf229
1 changed files with 197 additions and 148 deletions
|
|
@ -1,150 +1,175 @@
|
||||||
# Brainstorm: `claude-code` backend for end-to-end KTX ingest
|
# Brainstorm: `claude-code` backend with full KTX LLM parity
|
||||||
|
|
||||||
Adds a `claude-code` selection that makes **all `ktx ingest` LLM work** run
|
Adds a `claude-code` backend that gives KTX full parity with the existing
|
||||||
through `@anthropic-ai/claude-agent-sdk`, reusing the user's existing local
|
`ANTHROPIC_API_KEY`-based `anthropic` backend for **all KTX LLM calls**. The
|
||||||
Claude Code authentication. The user experience stays the same: users run
|
backend uses `@anthropic-ai/claude-agent-sdk` and reuses the user's existing
|
||||||
`ktx ingest`; the backend is selected in `ktx.yaml`.
|
local Claude Code authentication. Users select it in `ktx.yaml`.
|
||||||
|
|
||||||
This is not an implementation plan. It is the revised design after iterating on
|
This is not an implementation plan. It is the revised design after expanding
|
||||||
the brainstorm with the requirement that **all KTX ingest capabilities must work
|
the requirement from "`ktx ingest` works with Claude Code" to "every KTX LLM
|
||||||
with `claude-code`**. The follow-up implementation plan should be written
|
call works with Claude Code." The follow-up implementation plan should be
|
||||||
separately.
|
written separately.
|
||||||
|
|
||||||
## Core decision
|
## Core decision
|
||||||
|
|
||||||
`claude-code` is no longer only an agent-runner backend. It is an ingest-capable
|
`claude-code` is a first-class global LLM backend. Any code path that currently
|
||||||
LLM runtime that covers both kinds of LLM work used by ingest:
|
works with `llm.provider.backend: anthropic` must work with
|
||||||
|
`llm.provider.backend: claude-code`, unless it is not an LLM call at all.
|
||||||
|
|
||||||
- **Agent loops**: work-unit execution, reconciliation, context-candidate
|
This includes:
|
||||||
curation pagination, and memory-agent ingestion paths that call
|
|
||||||
`agentRunner.runLoop(...)`.
|
|
||||||
- **Non-agent generation**: page triage and light extraction, which currently
|
|
||||||
call `KtxLlmProvider` directly through `generateText`.
|
|
||||||
|
|
||||||
The implementation must not make page triage silently disappear when the user
|
- Agent loops implemented through `AgentRunnerService.runLoop(...)`.
|
||||||
chooses `claude-code`. Today `PageTriageService` is only constructed when
|
- Text generation through `generateKtxText(...)`.
|
||||||
`resolveAgentRunner(...)` returns an AI SDK `llmProvider`
|
- Structured object generation through `generateKtxObject(...)`.
|
||||||
(`packages/context/src/ingest/local-bundle-runtime.ts:684-693`). Under the new
|
- Local ingest and MCP-triggered local ingest flows.
|
||||||
design, ingest gets a generation runtime for `claude-code`, so page triage and
|
- Page triage and light extraction.
|
||||||
light extraction still run.
|
- Context-candidate curation and reconciliation.
|
||||||
|
- Memory capture.
|
||||||
|
- Scan/enrichment internals and relationship LLM proposals.
|
||||||
|
- Future KTX LLM call sites that use the shared runtime boundary.
|
||||||
|
|
||||||
|
Commands that do not use LLMs do not need special Claude Code behavior. There
|
||||||
|
must be no silent fallback from `claude-code` to gateway, Anthropic API-key
|
||||||
|
execution, or deterministic output.
|
||||||
|
|
||||||
## Goals
|
## Goals
|
||||||
|
|
||||||
- Let a KTX user run every `ktx ingest` mode against their existing local Claude
|
- Let a KTX user run all KTX LLM-backed behavior through their existing local
|
||||||
Code session without provisioning `ANTHROPIC_API_KEY`, Vertex credentials, or
|
Claude Code session without provisioning `ANTHROPIC_API_KEY`, Vertex
|
||||||
an AI Gateway key.
|
credentials, or an AI Gateway key.
|
||||||
- Cover scheduled pulls, upload ingest, Metabase fan-out, page triage, light
|
- Preserve the existing user-facing CLI and MCP behavior. `claude-code` changes
|
||||||
extraction, context-candidate curation, work-unit execution, reconciliation,
|
how LLM calls execute, not which KTX workflows exist.
|
||||||
memory capture invoked from ingest, and source-specific tools such as
|
- Preserve role-based model selection. `llm.models.default`, `triage`,
|
||||||
historic-SQL evidence emission.
|
`candidateExtraction`, `curator`, `reconcile`, and `repair` remain the source
|
||||||
- Preserve KTX's per-stage tool curation. Each stage exposes exactly the KTX
|
of model selection for every LLM call.
|
||||||
tools it already selected; Claude Code built-ins, filesystem-discovered MCP
|
- Preserve KTX's curated tool boundaries. Claude Code built-ins,
|
||||||
servers, hooks, skills, plugins, agents, and slash commands must not expand
|
filesystem-discovered MCP servers, hooks, skills, plugins, agents, and slash
|
||||||
the tool surface.
|
commands must not expand the tool surface for KTX agent loops.
|
||||||
- Keep embeddings independent. Claude does not provide embeddings; users keep
|
- Keep embeddings independent. Claude does not provide embeddings; users keep
|
||||||
configuring `ingest.embeddings` as they do today.
|
configuring `ingest.embeddings` and scan/enrichment embeddings as they do
|
||||||
|
today.
|
||||||
- Fail fast with a clear message if local Claude Code authentication is not
|
- Fail fast with a clear message if local Claude Code authentication is not
|
||||||
usable.
|
usable.
|
||||||
|
|
||||||
## Non-goals
|
## Non-goals
|
||||||
|
|
||||||
- **Non-ingest LLM surfaces.** The required target is end-to-end `ktx ingest`.
|
- **Embedding parity.** Embeddings remain separate from LLM execution.
|
||||||
Other internal LLM consumers are out of scope for this spec. The config and
|
- **Tool-call repair parity in the first pass.** The AI SDK runner uses
|
||||||
runtime design must still avoid accidental gateway fall-through when
|
|
||||||
`llm.provider.backend` is `claude-code`.
|
|
||||||
- **Tool-call repair parity.** The AI SDK runner uses
|
|
||||||
`experimental_repairToolCall` (`packages/llm/src/repair.ts:35-88`). The Claude
|
`experimental_repairToolCall` (`packages/llm/src/repair.ts:35-88`). The Claude
|
||||||
Agent SDK has no transparent same-step repair hook. MVP behavior is next-turn
|
Agent SDK has no transparent same-step repair hook. MVP behavior is next-turn
|
||||||
self-correction from schema errors or a normal tool-failure count.
|
self-correction from schema errors or a normal tool-failure count.
|
||||||
- **OTEL telemetry parity.** The AI SDK runner uses `experimental_telemetry`.
|
- **OTEL telemetry parity in the first pass.** The AI SDK runner uses
|
||||||
The Agent SDK exposes hooks such as `PostToolUseFailure` and `SessionEnd`, but
|
`experimental_telemetry`. The Agent SDK exposes hooks such as
|
||||||
no drop-in OTEL switch. MVP ships without telemetry parity on this backend.
|
`PostToolUseFailure` and `SessionEnd`, but no drop-in OTEL switch. MVP ships
|
||||||
|
without telemetry parity on this backend.
|
||||||
- **Productizing Claude subscription limits.** Documentation must frame this as
|
- **Productizing Claude subscription limits.** Documentation must frame this as
|
||||||
"use your own local Claude Code session," not as a third-party Claude Max or
|
"use your own local Claude Code session," not as a third-party Claude Max or
|
||||||
Claude.ai product feature.
|
Claude.ai product feature.
|
||||||
|
|
||||||
## Approaches considered
|
## Approaches considered
|
||||||
|
|
||||||
### Recommended: Ingest LLM runtime port
|
### Recommended: global LLM runtime port
|
||||||
|
|
||||||
Introduce a backend-neutral KTX LLM runtime port for operations, not just model
|
Introduce a backend-neutral KTX LLM runtime port for operations, not just model
|
||||||
construction:
|
construction:
|
||||||
|
|
||||||
```ts
|
```ts
|
||||||
interface KtxGenerationPort {
|
interface KtxLlmRuntimePort {
|
||||||
generateText(input: KtxGenerateTextInput): Promise<string>;
|
generateText(input: KtxGenerateTextInput): Promise<string>;
|
||||||
generateObject<T>(input: KtxGenerateObjectInput<T>): Promise<T>;
|
generateObject<T>(input: KtxGenerateObjectInput<T>): Promise<T>;
|
||||||
}
|
runAgentLoop(params: RunLoopParams): Promise<RunLoopResult>;
|
||||||
|
|
||||||
interface AgentRunnerPort {
|
|
||||||
runLoop(params: RunLoopParams): Promise<RunLoopResult>;
|
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
The existing AI SDK implementation adapts `KtxLlmProvider` to these ports. The
|
The existing `anthropic`, `vertex`, and `gateway` backends implement the runtime
|
||||||
new Claude Code implementation uses `query()` from
|
through the AI SDK and existing `KtxLlmProvider`. The new `claude-code` backend
|
||||||
`@anthropic-ai/claude-agent-sdk`. Ingest services depend on the ports:
|
implements the same runtime through `@anthropic-ai/claude-agent-sdk`.
|
||||||
|
|
||||||
- `PageTriageService` depends on `KtxGenerationPort`, not raw
|
This is the recommended approach because KTX call sites need operations:
|
||||||
`KtxLlmProvider`.
|
"generate text," "generate a structured object," and "run an agent loop." They
|
||||||
- `generateKtxText` / `generateKtxObject` become thin helpers over the
|
do not inherently need direct access to an AI SDK `LanguageModel`. The Agent SDK
|
||||||
generation port or move behind it.
|
is a session/agent API, not an AI SDK model factory, so the runtime port avoids
|
||||||
- `AgentRunnerService` and `ClaudeAgentSdkRunnerService` both implement
|
pretending those APIs are the same.
|
||||||
`AgentRunnerPort`.
|
|
||||||
|
|
||||||
This is the recommended approach because it matches the Agent SDK's actual
|
### Rejected: fake AI SDK `LanguageModel` for Claude Code
|
||||||
shape. The Agent SDK is an agent/session API, not an AI SDK `LanguageModel`
|
|
||||||
factory, so forcing it into `KtxLlmProvider.getModel(...)` would create a false
|
|
||||||
abstraction and leave page triage broken.
|
|
||||||
|
|
||||||
### Rejected: agent-runner-only backend
|
Trying to make Claude Code look like an AI SDK `LanguageModel` would be brittle.
|
||||||
|
The Agent SDK owns session execution, permissions, MCP tools, structured output,
|
||||||
|
and result messages. Those semantics do not map cleanly onto a normal
|
||||||
|
`getModel(...)` return value.
|
||||||
|
|
||||||
This was the previous version of the spec. It made work-unit and reconciliation
|
### Rejected: branch at every call site
|
||||||
agent loops possible, but it did not cover page triage or light extraction.
|
|
||||||
Because `ktx ingest` uses those non-agent LLM calls for document-like sources,
|
|
||||||
this does not satisfy the updated requirement.
|
|
||||||
|
|
||||||
### Rejected for MVP: Claude Code OpenAI proxy
|
Adding `if backend === "claude-code"` around each LLM call would work briefly
|
||||||
|
but would duplicate prompt wrapping, structured output handling, debug logging,
|
||||||
Using a proxy or `claude -p` subprocess would avoid some TypeScript adapter work,
|
tool conversion, auth checks, and error mapping. It would also make future LLM
|
||||||
but it would add another protocol boundary, make tool control harder to prove,
|
call sites easy to miss.
|
||||||
and move away from the official Agent SDK API.
|
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
```text
|
```text
|
||||||
ktx ingest
|
ktx.yaml
|
||||||
-> createLocalBundleIngestRuntime(...)
|
llm.provider.backend: anthropic | vertex | gateway | claude-code
|
||||||
-> resolveIngestLlmRuntime(...)
|
llm.models.<role>: model alias or model ID
|
||||||
-> AI SDK runtime
|
|
||||||
- KtxGenerationPort via generateText / Output.object
|
|
||||||
- AgentRunnerPort via current AgentRunnerService
|
|
||||||
-> Claude Code runtime
|
|
||||||
- KtxGenerationPort via Agent SDK query()
|
|
||||||
- AgentRunnerPort via ClaudeAgentSdkRunnerService
|
|
||||||
|
|
||||||
-> PageTriageService
|
createLocalKtxLlmRuntimeFromConfig(project.config.llm)
|
||||||
-> generation.generateText({ role: "triage", ... })
|
-> AiSdkKtxLlmRuntime
|
||||||
|
- wraps existing KtxLlmProvider
|
||||||
|
- generateText / Output.object / AgentRunnerService
|
||||||
|
-> ClaudeCodeKtxLlmRuntime
|
||||||
|
- uses @anthropic-ai/claude-agent-sdk query()
|
||||||
|
- implements text, object, and agent-loop operations
|
||||||
|
|
||||||
-> IngestBundleRunner stages
|
All KTX LLM call sites
|
||||||
-> agentRunner.runLoop({ modelRole, toolSet, stepBudget, ... })
|
-> KtxLlmRuntimePort
|
||||||
```
|
```
|
||||||
|
|
||||||
The runtime is selected once at the context-runtime DI boundary. The main ingest
|
The runtime is selected at the same boundaries that currently construct an
|
||||||
integration point remains `resolveAgentRunner` /
|
`llmProvider` or `AgentRunnerService`:
|
||||||
`createLocalBundleIngestRuntime` in
|
|
||||||
`packages/context/src/ingest/local-bundle-runtime.ts`, but the function should
|
|
||||||
evolve from "resolve an agent runner plus optional AI SDK provider" into
|
|
||||||
"resolve the ingest LLM runtime ports." The memory-agent construction path in
|
|
||||||
`packages/context/src/memory/local-memory.ts` needs the same port treatment.
|
|
||||||
|
|
||||||
`packages/cli/src/runtime.ts` is the Python-runtime command handler; it is not
|
- `packages/context/src/llm/local-config.ts`
|
||||||
the agent-runner or generation-runtime integration point.
|
- `packages/context/src/ingest/local-bundle-runtime.ts`
|
||||||
|
- `packages/context/src/memory/local-memory.ts`
|
||||||
|
- `packages/context/src/scan/local-scan.ts`
|
||||||
|
- `packages/context/src/mcp/local-project-ports.ts`
|
||||||
|
- Any CLI setup/status/doctor code that validates LLM readiness
|
||||||
|
|
||||||
|
After the change, services should not need to know whether the configured
|
||||||
|
backend is AI SDK based or Claude Code based. They call the runtime operation
|
||||||
|
they need.
|
||||||
|
|
||||||
|
## LLM call-site migration
|
||||||
|
|
||||||
|
The implementation plan must migrate every current KTX LLM call site to the
|
||||||
|
runtime port:
|
||||||
|
|
||||||
|
- `packages/context/src/llm/generation.ts`: `generateKtxText` and
|
||||||
|
`generateKtxObject` become runtime-backed helpers or are folded into the
|
||||||
|
runtime.
|
||||||
|
- `packages/context/src/agent/agent-runner.service.ts`: the AI SDK agent loop
|
||||||
|
becomes the AI SDK implementation of `runAgentLoop`.
|
||||||
|
- `packages/context/src/ingest/page-triage/page-triage.service.ts`: page triage
|
||||||
|
and light extraction depend on `KtxLlmRuntimePort`, not raw `KtxLlmProvider`.
|
||||||
|
- `packages/context/src/scan/description-generation.ts`: AI descriptions use
|
||||||
|
the runtime text-generation operation.
|
||||||
|
- `packages/context/src/scan/relationship-llm-proposal.ts`: relationship
|
||||||
|
proposals use the runtime object-generation operation.
|
||||||
|
- `packages/context/src/ingest/stages/stage-3-work-units.ts`,
|
||||||
|
`packages/context/src/ingest/stages/stage-4-reconciliation.ts`,
|
||||||
|
`packages/context/src/ingest/context-candidates/curator-pagination.service.ts`,
|
||||||
|
and `packages/context/src/memory/memory-agent.service.ts`: agent loops use the
|
||||||
|
runtime agent-loop operation or a thin `AgentRunnerPort` backed by it.
|
||||||
|
- Test helpers and MCP local project ports that inject `llmProvider` or
|
||||||
|
`agentRunner` must either inject the runtime port or use compatibility test
|
||||||
|
adapters during the migration.
|
||||||
|
|
||||||
|
The plan must include a grep-based audit so new or overlooked `getModel(...)`,
|
||||||
|
`generateKtxText(...)`, `generateKtxObject(...)`, `AgentRunnerService`, and
|
||||||
|
`llmProvider` usages are either migrated or explicitly proven non-runtime.
|
||||||
|
|
||||||
## Config design
|
## Config design
|
||||||
|
|
||||||
The plan should make `claude-code` a first-class config value, not a hidden
|
The config should make `claude-code` a first-class backend:
|
||||||
side-channel. Recommended shape:
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
llm:
|
llm:
|
||||||
|
|
@ -156,28 +181,28 @@ llm:
|
||||||
candidateExtraction: sonnet
|
candidateExtraction: sonnet
|
||||||
curator: sonnet
|
curator: sonnet
|
||||||
reconcile: sonnet
|
reconcile: sonnet
|
||||||
|
repair: sonnet
|
||||||
```
|
```
|
||||||
|
|
||||||
Implementation implications:
|
Implementation implications:
|
||||||
|
|
||||||
- Extend `KTX_LLM_BACKENDS` in `packages/context/src/project/config.ts` and
|
- Extend `KTX_LLM_BACKENDS` in `packages/context/src/project/config.ts` and
|
||||||
`KtxLlmBackend` in `packages/llm/src/types.ts`.
|
`KtxLlmBackend` in `packages/llm/src/types.ts`.
|
||||||
- Update setup, status, doctor, and local provider resolution so
|
- Update setup, status, doctor, schema generation, examples, and docs so
|
||||||
`claude-code` does not fall through to `gateway`.
|
`claude-code` is understood everywhere `anthropic` is understood.
|
||||||
- For `claude-code`, do not construct a fake AI SDK `LanguageModel`. Construct
|
- Update `createKtxLlmProvider` / `createModelFactory` so unsupported backend
|
||||||
the Claude Code generation/runtime ports.
|
values throw instead of falling through to gateway.
|
||||||
- Keep `llm.models` as the per-role binding source. The Claude Code runtime maps
|
- Keep `llm.models` as the per-role binding source. The Claude Code runtime maps
|
||||||
each KTX role to the configured model string for the current call. The plan
|
each KTX role to the configured model string for the current call.
|
||||||
must decide and test the accepted model aliases, for example `sonnet`,
|
- Define accepted model aliases, such as `sonnet`, `opus`, and `haiku`, and full
|
||||||
`opus`, `haiku`, or full Claude model IDs supported by the SDK.
|
model IDs supported by the pinned SDK version.
|
||||||
- If non-ingest code sees `backend: claude-code` before it has been ported, it
|
|
||||||
must fail fast with a clear unsupported-backend message. It must not silently
|
|
||||||
route to gateway.
|
|
||||||
|
|
||||||
## Claude Agent SDK runtime behavior
|
## Claude Agent SDK runtime behavior
|
||||||
|
|
||||||
Every Agent SDK call must be isolated and deterministic enough for KTX ingest.
|
Every Agent SDK call must be isolated enough for KTX execution. Use explicit
|
||||||
Use explicit options even when SDK defaults currently match the desired value.
|
options even when SDK defaults currently match the desired value.
|
||||||
|
|
||||||
|
For agent loops with tools:
|
||||||
|
|
||||||
```ts
|
```ts
|
||||||
query({
|
query({
|
||||||
|
|
@ -213,33 +238,43 @@ query({
|
||||||
});
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
|
For plain text generation:
|
||||||
|
|
||||||
|
- Use the same `query()` runtime with `maxTurns: 1`.
|
||||||
|
- Pass `settingSources: []`, `skills: []`, `plugins: []`, `tools: []`, and
|
||||||
|
`permissionMode: "dontAsk"`.
|
||||||
|
- Do not expose MCP tools unless the KTX call explicitly passed tools.
|
||||||
|
- Return the final result message text.
|
||||||
|
|
||||||
|
For structured object generation:
|
||||||
|
|
||||||
|
- Use the Agent SDK structured output option for JSON schema output.
|
||||||
|
- Convert KTX Zod schemas at the runtime boundary.
|
||||||
|
- Parse and validate the returned object with the original KTX schema before
|
||||||
|
returning it to the caller.
|
||||||
|
|
||||||
The plan must confirm the exact option names against the pinned SDK version, but
|
The plan must confirm the exact option names against the pinned SDK version, but
|
||||||
the required outcome is fixed:
|
the required outcome is fixed:
|
||||||
|
|
||||||
- Filesystem settings are not loaded. `settingSources: []` is explicit, and the
|
- Filesystem settings are not loaded. `settingSources: []` is explicit, and the
|
||||||
implementation should assert from the SDK init message that no unexpected
|
implementation should assert from the SDK init message that no unexpected
|
||||||
settings-derived commands, skills, agents, or MCP servers are active.
|
settings-derived commands, skills, agents, plugins, or MCP servers are active.
|
||||||
- Skills are disabled with `skills: []`, and plugins are disabled with
|
- Skills are disabled with `skills: []`, and plugins are disabled with
|
||||||
`plugins: []`.
|
`plugins: []`.
|
||||||
- Only KTX MCP tools are available and auto-approved. `allowedTools` alone is
|
- `allowedTools` alone is not sufficient because the current SDK docs describe
|
||||||
not sufficient because the current SDK docs describe it as auto-approval, not
|
it as auto-approval, not restriction. Use `tools`, `permissionMode:
|
||||||
restriction. Use `tools`, `permissionMode: "dontAsk"`, and explicit
|
"dontAsk"`, and explicit `disallowedTools` for built-ins.
|
||||||
`disallowedTools` for built-ins.
|
|
||||||
- Built-ins are denied even if a future SDK default changes.
|
- Built-ins are denied even if a future SDK default changes.
|
||||||
- `cwd` is `project.projectDir`, resolved at startup via `resolveKtxProjectDir`,
|
- `cwd` is `project.projectDir`, resolved at startup via `resolveKtxProjectDir`,
|
||||||
not `process.cwd()`.
|
not `process.cwd()`.
|
||||||
- Sessions are not persisted for ingest unless the plan identifies a concrete
|
- Sessions are not persisted unless the plan identifies a concrete debugging
|
||||||
debugging feature that needs persistence.
|
feature that needs persistence.
|
||||||
|
|
||||||
For non-agent text generation, use the same isolated runtime with no MCP tools,
|
|
||||||
`maxTurns: 1`, and no filesystem settings. For structured outputs, use the Agent
|
|
||||||
SDK's JSON-schema output format and convert KTX's Zod schemas at the boundary.
|
|
||||||
|
|
||||||
## Tool boundary
|
## Tool boundary
|
||||||
|
|
||||||
The final `RunLoopParams.toolSet` cannot remain a raw AI SDK `Record<string,
|
Agent-loop tools cannot remain only raw AI SDK `Record<string, Tool>` values if
|
||||||
Tool>` if two backends must consume it. The plan must define a backend-neutral
|
two backends must consume them. The plan must define a backend-neutral tool
|
||||||
tool descriptor for the **final** tool map handed to the runner:
|
descriptor for the final tool map handed to an agent loop:
|
||||||
|
|
||||||
```ts
|
```ts
|
||||||
interface KtxRuntimeToolDescriptor<TInput, TOutput> {
|
interface KtxRuntimeToolDescriptor<TInput, TOutput> {
|
||||||
|
|
@ -254,10 +289,8 @@ Every composed tool entry must preserve the descriptor, including:
|
||||||
|
|
||||||
- `BaseTool` outputs from factory toolsets.
|
- `BaseTool` outputs from factory toolsets.
|
||||||
- Source-specific raw tools such as `emit_historic_sql_evidence` in
|
- Source-specific raw tools such as `emit_historic_sql_evidence` in
|
||||||
`packages/context/src/ingest/local-bundle-runtime.ts:543-556`.
|
`packages/context/src/ingest/local-bundle-runtime.ts`.
|
||||||
- Stage-local tools in `buildWuToolSet` and `buildReconcileToolSet`
|
- Stage-local tools in `buildWuToolSet` and `buildReconcileToolSet`.
|
||||||
(`packages/context/src/ingest/stages/build-wu-context.ts`,
|
|
||||||
`packages/context/src/ingest/stages/build-reconcile-context.ts`).
|
|
||||||
- Inline `load_skill`, read/raw/span, stage/diff, eviction, and emit tools in
|
- Inline `load_skill`, read/raw/span, stage/diff, eviction, and emit tools in
|
||||||
`packages/context/src/ingest/ingest-bundle.runner.ts`.
|
`packages/context/src/ingest/ingest-bundle.runner.ts`.
|
||||||
- Memory-agent `load_skill` in
|
- Memory-agent `load_skill` in
|
||||||
|
|
@ -267,8 +300,8 @@ Every composed tool entry must preserve the descriptor, including:
|
||||||
The AI SDK adapter converts descriptors to `tool(...)`. The Claude Code adapter
|
The AI SDK adapter converts descriptors to `tool(...)`. The Claude Code adapter
|
||||||
converts descriptors to Agent SDK `tool(name, description, schema.shape,
|
converts descriptors to Agent SDK `tool(name, description, schema.shape,
|
||||||
handler)` entries inside `createSdkMcpServer(...)`. KTX tool handlers return
|
handler)` entries inside `createSdkMcpServer(...)`. KTX tool handlers return
|
||||||
`{ markdown, structured }`; the Claude adapter returns the markdown as text
|
`{ markdown, structured }`; the Claude adapter returns markdown as text content
|
||||||
content and may include structured JSON in the text only if a caller needs it.
|
and may include structured JSON only if a caller needs it.
|
||||||
|
|
||||||
Non-object schemas are unsupported for `claude-code` and must be rejected at
|
Non-object schemas are unsupported for `claude-code` and must be rejected at
|
||||||
startup with a clear error. In practice KTX tool inputs are already `z.object`.
|
startup with a clear error. In practice KTX tool inputs are already `z.object`.
|
||||||
|
|
@ -287,10 +320,14 @@ carry the terminal reason. They remain useful for lifecycle logging. Tool failur
|
||||||
counting should use `PostToolUseFailure` and feed the same mechanism that
|
counting should use `PostToolUseFailure` and feed the same mechanism that
|
||||||
`stage-3-work-units.ts` checks through `toolFailureCount?(wu.unitKey)`.
|
`stage-3-work-units.ts` checks through `toolFailureCount?(wu.unitKey)`.
|
||||||
|
|
||||||
|
For text and object generation, SDK authentication, billing, rate-limit,
|
||||||
|
permission, max-turn, structured-output, and execution errors must map to the
|
||||||
|
same error surfaces that KTX uses for the Anthropic API-key backend.
|
||||||
|
|
||||||
## Auth and setup
|
## Auth and setup
|
||||||
|
|
||||||
`ktx setup` must validate that Claude Code SDK auth is usable, not just that
|
`ktx setup`, status, and doctor flows must validate that Claude Code SDK auth is
|
||||||
`~/.claude/` exists. Acceptable validation strategies:
|
usable, not just that `~/.claude/` exists. Acceptable validation strategies:
|
||||||
|
|
||||||
- A minimal SDK probe call with `settingSources: []`, `tools: []`, and
|
- A minimal SDK probe call with `settingSources: []`, `tools: []`, and
|
||||||
`maxTurns: 1`.
|
`maxTurns: 1`.
|
||||||
|
|
@ -299,17 +336,18 @@ counting should use `PostToolUseFailure` and feed the same mechanism that
|
||||||
state that it proves auth usability.
|
state that it proves auth usability.
|
||||||
|
|
||||||
Failure copy should tell the user to authenticate Claude Code locally with the
|
Failure copy should tell the user to authenticate Claude Code locally with the
|
||||||
Claude Code CLI, then rerun setup or ingest.
|
Claude Code CLI, then rerun setup or the command they attempted.
|
||||||
|
|
||||||
## Documentation impact
|
## Documentation impact
|
||||||
|
|
||||||
Docs updates are required because this changes user-visible setup and ingest
|
Docs updates are required because this changes user-visible setup and LLM
|
||||||
behavior:
|
provider behavior:
|
||||||
|
|
||||||
- `docs-site/content/docs/getting-started/quickstart.mdx`
|
- `docs-site/content/docs/getting-started/quickstart.mdx`
|
||||||
- `docs-site/content/docs/cli-reference/ktx-setup.mdx`
|
- `docs-site/content/docs/cli-reference/ktx-setup.mdx`
|
||||||
- `docs-site/content/docs/guides/building-context.mdx`
|
- `docs-site/content/docs/guides/building-context.mdx`
|
||||||
- Any config reference page that documents `llm.provider.backend`
|
- Any config reference page that documents `llm.provider.backend`
|
||||||
|
- Any status or doctor docs that describe LLM readiness
|
||||||
|
|
||||||
The docs must say that `claude-code` uses the user's own local Claude Code
|
The docs must say that `claude-code` uses the user's own local Claude Code
|
||||||
session. Do not describe it as a way for KTX to resell, pool, or productize
|
session. Do not describe it as a way for KTX to resell, pool, or productize
|
||||||
|
|
@ -322,17 +360,24 @@ Claude subscription limits.
|
||||||
(`packages/llm/src/types.ts`, `packages/llm/src/model-provider.ts`).
|
(`packages/llm/src/types.ts`, `packages/llm/src/model-provider.ts`).
|
||||||
- Project config currently accepts `llm.provider.backend: none | anthropic |
|
- Project config currently accepts `llm.provider.backend: none | anthropic |
|
||||||
vertex | gateway` (`packages/context/src/project/config.ts`).
|
vertex | gateway` (`packages/context/src/project/config.ts`).
|
||||||
- `resolveAgentRunner(...)` currently requires an AI SDK `llmProvider`, and
|
- `generateKtxText` and `generateKtxObject` are shared non-agent generation
|
||||||
page triage is only constructed when that provider exists
|
helpers (`packages/context/src/llm/generation.ts`).
|
||||||
(`packages/context/src/ingest/local-bundle-runtime.ts`).
|
- `AgentRunnerService` is the shared AI SDK agent-loop implementation
|
||||||
- Page triage and light extraction are non-agent LLM calls using
|
(`packages/context/src/agent/agent-runner.service.ts`).
|
||||||
`llmProvider.getModel("triage")` and AI SDK `generateText`
|
- Page triage and light extraction currently use raw `KtxLlmProvider`
|
||||||
(`packages/context/src/ingest/page-triage/page-triage.service.ts`).
|
(`packages/context/src/ingest/page-triage/page-triage.service.ts`).
|
||||||
- The Agent SDK TypeScript reference documents `settingSources` defaulting to
|
- Scan/enrichment internals currently use `createLocalKtxLlmProviderFromConfig`,
|
||||||
no filesystem settings, `allowedTools` as auto-approval rather than
|
`generateKtxText`, and `generateKtxObject`
|
||||||
restriction, `permissionMode: "dontAsk"`, `tools`, `disallowedTools`,
|
(`packages/context/src/scan/local-scan.ts`,
|
||||||
`maxTurns`, `mcpServers`, `cwd`, `persistSession`, and SDK result/hook
|
`packages/context/src/scan/description-generation.ts`,
|
||||||
message shapes.
|
`packages/context/src/scan/relationship-llm-proposal.ts`).
|
||||||
|
- Local ingest and MCP local project ports inject `llmProvider` and
|
||||||
|
`agentRunner` today (`packages/context/src/ingest/local-bundle-runtime.ts`,
|
||||||
|
`packages/context/src/mcp/local-project-ports.ts`).
|
||||||
|
- The Agent SDK TypeScript reference documents `settingSources` defaulting to no
|
||||||
|
filesystem settings, `allowedTools` as auto-approval rather than restriction,
|
||||||
|
`permissionMode: "dontAsk"`, `tools`, `disallowedTools`, `maxTurns`,
|
||||||
|
`mcpServers`, `cwd`, `persistSession`, and SDK result/hook message shapes.
|
||||||
- The Agent SDK MCP docs show registering MCP servers in `query()` options and
|
- The Agent SDK MCP docs show registering MCP servers in `query()` options and
|
||||||
using `allowedTools` for MCP tool access.
|
using `allowedTools` for MCP tool access.
|
||||||
- The Agent SDK skills docs say discovered skills can be controlled with the
|
- The Agent SDK skills docs say discovered skills can be controlled with the
|
||||||
|
|
@ -343,13 +388,17 @@ Claude subscription limits.
|
||||||
|
|
||||||
1. Confirm exact TypeScript option names and result-message discriminants
|
1. Confirm exact TypeScript option names and result-message discriminants
|
||||||
against the pinned `@anthropic-ai/claude-agent-sdk` version.
|
against the pinned `@anthropic-ai/claude-agent-sdk` version.
|
||||||
2. Define the final `KtxGenerationPort` and `AgentRunnerPort` file locations and
|
2. Define the final `KtxLlmRuntimePort` file location and package exports.
|
||||||
package exports.
|
|
||||||
3. Define model alias validation for `sonnet`, `opus`, `haiku`, and full model
|
3. Define model alias validation for `sonnet`, `opus`, `haiku`, and full model
|
||||||
IDs.
|
IDs.
|
||||||
4. Define the auth probe and make setup/status/doctor report actionable
|
4. Define the auth probe and make setup/status/doctor report actionable
|
||||||
messages.
|
messages.
|
||||||
5. Write tests that prove page triage is constructed and called under
|
5. Run a repo-wide audit for all LLM call sites and migrate each one to the
|
||||||
`llm.provider.backend: claude-code`.
|
runtime boundary.
|
||||||
6. Write tests that prove a raw built-in Claude Code tool request is denied and
|
6. Write tests proving `claude-code` works for text generation, structured
|
||||||
only `mcp__ktx__*` tools are available during ingest.
|
object generation, and agent-loop execution.
|
||||||
|
7. Write tests proving page triage, scan/enrichment internals, memory capture,
|
||||||
|
MCP-triggered local ingest, and normal local ingest all use the
|
||||||
|
`claude-code` runtime when configured.
|
||||||
|
8. Write tests proving a raw built-in Claude Code tool request is denied and
|
||||||
|
only `mcp__ktx__*` tools are available during KTX agent loops.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue