ktx/packages/cli/src/context/llm/local-config.ts
Andrey Avtomonov 494618ab14
feat: add codex llm backend for ktx runtime work (#253)
* feat: add codex sdk runner foundation

* feat: parse codex runtime events

* feat: expose codex runtime mcp tools

* feat: add codex llm runtime

* feat: wire codex llm backend

* test: avoid Array.fromAsync in codex runner test

* docs: document codex llm backend

* fix: tighten codex runtime config ownership

* fix: use codex sdk env and thread options

* fix: parse codex sdk event shapes

* test: add codex backend live smoke

* docs: clarify codex backend isolation

* fix: drive codex loop metrics from mcp events

* fix: enforce codex local step budget

* docs: disclose codex isolation limits

* fix: count all codex agent steps and stream step callbacks live

The agent-loop step budget only counted completed mcp_tool_call items, so
built-in command_execution steps (which the public Codex SDK/CLI surface can
still expose) never decremented the budget, letting ingest/reconciliation run
past stepBudget until Codex stopped on its own. onStepFinish was also replayed
only after the whole stream drained, so live work_unit_step / reconciliation
progress appeared stuck until the Codex process exited.

collectEvents is now the single live step accumulator: it counts every
completed agent-action item via a shared isCompletedAgentStep predicate
(command_execution, mcp_tool_call, file_change, web_search), fires onStepFinish
as each step completes, and enforces the budget on that broader count. A
no-tool turn still counts as one step. toolFailures stays MCP-specific, since a
non-zero command exit is normal agent exploration, not a loop failure.

* test: align ingest llm-guard assertions with codex backend

The skip-llm ingest guard message now lists codex as a valid backend and
mentions a Claude Code/Codex session plus a codex setup hint, but this slow
suite test still asserted the pre-codex wording. Update it to match the
production message (already covered by the local-bundle-runtime unit test) and
add the codex setup-line assertion.

* fix: treat codex error:null tool calls as success

The Codex SDK serializes error: null on successful mcp_tool_call items, so
the failure check (item.error !== undefined) flagged every successful tool
call as failed with the empty-payload default "Codex turn failed". This
killed every ingest work unit under the codex backend before it could
produce a patch.

Key on status === 'failed' (authoritative, always set) and only treat a
populated error object as a failure. Add a regression test built from a
verbatim real-SDK event capture.

* fix: default codex backend to gpt-5.5 and report real probe errors

The previous default gpt-5.3-codex is an API-key-only model that the OpenAI
API rejects under ChatGPT-account (subscription) auth, so codex status/setup
failed with a misleading "authentication is not usable" message even though
auth was fine.

- Default codex model is now gpt-5.5 (works on both subscription and API-key
  auth); the curated setup picker offers gpt-5.5 / gpt-5.4 / gpt-5.4-mini and
  keeps free-form entry for account-specific ids (e.g. gpt-5.3-codex-spark).
- runCodexAuthProbe now distinguishes "model not available" from an auth
  failure and surfaces the real API error: collectEvents retains stream
  events when the SDK throws on a non-zero exit, and the API error JSON
  envelope is unwrapped to its human-readable message.
- The Codex isolation warning now renders inside the clack setup frame.
- Docs updated to gpt-5.5 with a note that *-codex ids require API-key auth.

* fix: require llm.models.default in status and match codex probe remediation

Status reported a project ready when a non-none LLM backend was configured
without llm.models.default, but the runtime (resolveModelSlots) hard-requires
it, so ingest/scan/memory threw after `ktx status` said the project was usable.
buildLlmStatus now fails for any non-none backend missing models.default and no
longer invents a fallback model for claude-code/codex.

Codex probe failures now carry a category-matched fix: a model-access failure
steers the user at llm.models.default instead of the auth/install remediation.
runCodexAuthProbe returns the fix and status consumes it; the message stays
self-sufficient so setup output is unchanged.

Docs: README now lists the codex backend and local Codex auth; ktx-setup.mdx
states --llm-model only accepts codex/default or gpt-*/codex-* ids.

Repaired four doctor fixtures that configured a backend without models.default
(the now-correctly-blocked config) and added coverage for the new behavior.
2026-06-02 13:57:11 +02:00

194 lines
6.8 KiB
TypeScript

import { createKtxEmbeddingProvider } from '../../llm/embedding-provider.js';
import { createKtxLlmProvider } from '../../llm/model-provider.js';
import type { KtxEmbeddingConfig, KtxEmbeddingProvider, KtxLlmConfig, KtxLlmProvider, KtxModelRole } from '../../llm/types.js';
import { resolveKtxConfigReference } from '../core/config-reference.js';
import type { KtxProjectEmbeddingConfig, KtxProjectLlmConfig } from '../project/config.js';
import { AiSdkKtxLlmRuntime } from './ai-sdk-runtime.js';
import { ClaudeCodeKtxLlmRuntime } from './claude-code-runtime.js';
import { CodexKtxLlmRuntime } from './codex-runtime.js';
import type { KtxLlmRuntimePort } from './runtime-port.js';
interface LocalConfigDeps {
env?: NodeJS.ProcessEnv;
projectDir?: string;
createKtxLlmProvider?: typeof createKtxLlmProvider;
createKtxEmbeddingProvider?: typeof createKtxEmbeddingProvider;
createClaudeCodeRuntime?: (deps: ConstructorParameters<typeof ClaudeCodeKtxLlmRuntime>[0]) => KtxLlmRuntimePort;
createCodexRuntime?: (deps: ConstructorParameters<typeof CodexKtxLlmRuntime>[0]) => KtxLlmRuntimePort;
createAiSdkRuntime?: (deps: { llmProvider: KtxLlmProvider }) => KtxLlmRuntimePort;
}
function resolveOptional(value: string | undefined, env: NodeJS.ProcessEnv): string | undefined {
return resolveKtxConfigReference(value, env) || undefined;
}
function resolveRequired(value: string | undefined, env: NodeJS.ProcessEnv, message: string): string {
const resolved = resolveOptional(value, env);
if (!resolved) {
throw new Error(message);
}
return resolved;
}
function resolveModelSlots(
models: KtxProjectLlmConfig['models'],
env: NodeJS.ProcessEnv,
): KtxLlmConfig['modelSlots'] {
const resolved: Partial<Record<KtxModelRole, string>> & { default?: string } = {};
for (const [role, value] of Object.entries(models)) {
if (value) {
resolved[role as KtxModelRole] = resolveRequired(value, env, `llm.models.${role} is required`);
}
}
if (!resolved.default) {
throw new Error('llm.models.default is required when llm.provider.backend is not none');
}
return resolved as KtxLlmConfig['modelSlots'];
}
function resolvedProviderConfig(
config: { api_key?: string; base_url?: string } | undefined,
env: NodeJS.ProcessEnv,
): { apiKey?: string; baseURL?: string } | undefined {
if (!config) {
return undefined;
}
const apiKey = resolveOptional(config.api_key, env);
const baseURL = resolveOptional(config.base_url, env);
if (!apiKey && !baseURL) {
return undefined;
}
return {
...(apiKey ? { apiKey } : {}),
...(baseURL ? { baseURL } : {}),
};
}
function resolvedVertexConfig(
config: { project?: string; location?: string } | undefined,
env: NodeJS.ProcessEnv,
): { project?: string; location: string } | undefined {
if (!config) {
return undefined;
}
const project = resolveOptional(config.project, env);
const location = resolveRequired(config.location, env, 'llm.provider.vertex.location is required');
return {
...(project ? { project } : {}),
location,
};
}
export function resolveLocalKtxLlmConfig(config: KtxProjectLlmConfig, env: NodeJS.ProcessEnv): KtxLlmConfig | null {
if (config.provider.backend === 'none') {
return null;
}
const modelSlots = resolveModelSlots(config.models, env);
const vertex = config.provider.backend === 'vertex' ? resolvedVertexConfig(config.provider.vertex, env) : undefined;
const anthropic = resolvedProviderConfig(config.provider.anthropic, env);
const gateway = resolvedProviderConfig(config.provider.gateway, env);
return {
backend: config.provider.backend,
...(vertex ? { vertex } : {}),
...(anthropic ? { anthropic } : {}),
...(gateway ? { gateway } : {}),
modelSlots,
promptCaching: config.promptCaching,
};
}
/** @internal */
export function createLocalKtxLlmProviderFromConfig(
config: KtxProjectLlmConfig,
deps: LocalConfigDeps = {},
): KtxLlmProvider | null {
const resolved = resolveLocalKtxLlmConfig(config, deps.env ?? process.env);
if (!resolved || resolved.backend === 'claude-code' || resolved.backend === 'codex') {
return null;
}
return (deps.createKtxLlmProvider ?? createKtxLlmProvider)(resolved);
}
export function createLocalKtxLlmRuntimeFromConfig(
config: KtxProjectLlmConfig,
deps: LocalConfigDeps = {},
): KtxLlmRuntimePort | null {
const resolved = resolveLocalKtxLlmConfig(config, deps.env ?? process.env);
if (!resolved) {
return null;
}
if (resolved.backend === 'claude-code') {
const projectDir = deps.projectDir;
if (!projectDir) {
throw new Error('projectDir is required when creating the claude-code LLM runtime');
}
return (deps.createClaudeCodeRuntime ?? ((runtimeDeps) => new ClaudeCodeKtxLlmRuntime(runtimeDeps)))({
projectDir,
modelSlots: resolved.modelSlots,
env: deps.env,
});
}
if (resolved.backend === 'codex') {
const projectDir = deps.projectDir;
if (!projectDir) {
throw new Error('projectDir is required when creating the codex LLM runtime');
}
return (deps.createCodexRuntime ?? ((runtimeDeps) => new CodexKtxLlmRuntime(runtimeDeps)))({
projectDir,
modelSlots: resolved.modelSlots,
});
}
const llmProvider = (deps.createKtxLlmProvider ?? createKtxLlmProvider)(resolved);
return (deps.createAiSdkRuntime ?? ((runtimeDeps) => new AiSdkKtxLlmRuntime(runtimeDeps)))({ llmProvider });
}
export function resolveLocalKtxEmbeddingConfig(
config: KtxProjectEmbeddingConfig,
env: NodeJS.ProcessEnv,
): KtxEmbeddingConfig | null {
if (config.backend === 'none') {
return null;
}
if (config.backend === 'sentence-transformers') {
const baseURL = config.sentenceTransformers?.base_url;
if (!baseURL) {
return null;
}
return {
backend: config.backend,
model: config.model ?? 'all-MiniLM-L6-v2',
dimensions: config.dimensions,
sentenceTransformers: {
baseURL,
pathPrefix: config.sentenceTransformers?.pathPrefix,
},
batchSize: config.batchSize,
};
}
if (config.backend === 'openai') {
const openai = resolvedProviderConfig(config.openai, env);
if (!openai?.apiKey) {
return null;
}
return {
backend: config.backend,
model: config.model ?? 'text-embedding-3-small',
dimensions: config.dimensions,
openai,
batchSize: config.batchSize,
};
}
throw new Error(`Unsupported KTX embedding backend: ${String((config as { backend?: string }).backend)}`);
}
/** @internal */
export function createLocalKtxEmbeddingProviderFromConfig(
config: KtxProjectEmbeddingConfig,
deps: LocalConfigDeps = {},
): KtxEmbeddingProvider | null {
const resolved = resolveLocalKtxEmbeddingConfig(config, deps.env ?? process.env);
return resolved ? (deps.createKtxEmbeddingProvider ?? createKtxEmbeddingProvider)(resolved) : null;
}