ktx/docs/superpowers/plans/2026-05-11-managed-agent-mcp-semantic-runtime.md
Andrey Avtomonov 9dad936ac7
feat: npm-managed Python runtime for @kaelio/ktx (#7)
* docs: add npm managed python runtime design

* build: add bundled python runtime wheel builder

* build: make local embedding dependencies optional

* build: bundle python runtime wheel in cli artifacts

* build: track bundled python runtime release artifact

* test: verify bundled python runtime wheel

* docs: add plan for bundled python runtime wheel

* test: cover managed python runtime lifecycle

* feat: add managed python runtime installer

* feat: add runtime command runner

* feat: expose runtime management commands

* test: verify managed python runtime commands

* docs: add plan for managed python runtime installer

* feat: add managed python command helper

* feat: use managed runtime for sl query compute

* feat: route sl query managed runtime policy

* docs: add plan for managed runtime sl query integration

* feat: add managed runtime daemon metadata

* feat: manage python daemon lifecycle

* feat: add runtime daemon start stop commands

* fix: verify managed runtime daemon lifecycle

* docs: add plan for managed runtime daemon lifecycle

* feat: add managed local embeddings config marker

* feat: add managed local embeddings daemon helper

* feat: use managed runtime for local embedding setup

* feat: pass managed runtime policy through setup

* docs: add plan for managed local embeddings runtime

* feat: read CLI package metadata dynamically

* feat: assemble public kaelio ktx npm package

* feat: release one public kaelio ktx npm artifact

* test: cover public kaelio ktx package invocations

* chore: verify public kaelio ktx package artifacts

* docs: add plan for public kaelio ktx npm package

* test: verify managed runtime in public package smoke

* test: finalize managed runtime release smoke

* docs: add plan for managed runtime release smoke

* test: specify local embeddings release smoke

* feat: add local embeddings runtime smoke

* chore: register local embeddings smoke

* fix: verify local embeddings smoke

* fix: restore artifact smoke python env helper

* docs: add plan for managed local embeddings release smoke

* refactor: share managed runtime install policy parsing

* feat: use managed runtime for agent semantic queries

* feat: use managed runtime for MCP semantic compute

* docs: add plan for managed agent and MCP semantic runtime

* feat(cli): add managed daemon HTTP helpers

* feat(cli): route local adapters through managed daemon

* feat(cli): use managed daemon for ingest helpers

* feat(cli): pass managed daemon options to scan

* feat(context): pass MCP ingest pull config options

* feat(cli): pass managed daemon options to serve ingest

* test: verify managed local ingest daemon runtime

* docs: add plan for managed local ingest daemon runtime

* docs: align managed runtime examples

* docs: add plan for managed runtime docs cleanup

* test: cover published package runtime smoke commands

* test: validate published package smoke outputs

* docs: add plan for published package runtime smoke

* build: stamp public npm package version

* release: add npm public release policy

* release: add guarded npm publish script

* release: document public npm release handoff

* docs: add plan for public npm release handoff

* test: cover managed runtime prune in package smoke

* docs: document managed runtime prune

* docs: add plan for managed runtime prune smoke and docs

* chore: encode uv runtime prerequisite policy

* fix: clarify missing uv runtime error

* docs: document uv runtime prerequisite

* docs: add plan for uv runtime prerequisite contract

* refactor: limit release artifacts to public package runtime

* chore: align release policy with bundled runtime wheel

* docs: describe single public runtime artifact surface

* test: verify single public runtime artifact contract

* docs: add plan for single public runtime artifact cleanup

* fix: align local embeddings smoke with public version

* docs: add plan for local embeddings smoke public version

* release: soft-launch as @kaelio/ktx@0.1.0-rc.0 on next tag

Publish target moves to the pre-release version 0.1.0-rc.0 under the next
dist-tag so npm install @kaelio/ktx (which resolves to latest) does not
pick up the soft-launch build. Users opt in via @kaelio/ktx@next.

* Fix release script boundary checks

* Remove PostHog from public package bundle
2026-05-11 15:50:34 +02:00

33 KiB

Managed Agent and MCP Semantic Runtime Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Make hidden agent semantic queries and MCP semantic compute use the KTX-managed core Python runtime instead of relying on a user-provided python -m ktx_daemon.

Architecture: Reuse the existing managed runtime command helper so every CLI semantic compute surface resolves the same bundled ktx-daemon executable. Keep explicit HTTP daemon URLs working for ktx serve --semantic-compute-url, and add runtime install policy flags where commands can lazily install the core runtime.

Tech Stack: TypeScript, Commander, Vitest, KTX CLI managed Python runtime, @ktx/context/daemon.


Existing status

This plan is based on docs/superpowers/specs/2026-05-11-npm-managed-python-runtime-design.md.

The following plans are based on that spec and are already implemented in this worktree:

  • docs/superpowers/plans/2026-05-11-bundled-python-runtime-wheel.md
  • docs/superpowers/plans/2026-05-11-managed-python-runtime-installer.md
  • docs/superpowers/plans/2026-05-11-managed-python-runtime-command-integration.md
  • docs/superpowers/plans/2026-05-11-managed-python-runtime-daemon-lifecycle.md
  • docs/superpowers/plans/2026-05-11-managed-local-embeddings-runtime.md
  • docs/superpowers/plans/2026-05-11-public-kaelio-ktx-npm-package.md
  • docs/superpowers/plans/2026-05-11-managed-python-runtime-release-smoke.md
  • docs/superpowers/plans/2026-05-11-managed-local-embeddings-release-smoke.md

Implementation evidence found before writing this plan includes:

  • scripts/build-python-runtime-wheel.mjs and packages/cli/assets/python/manifest.json.
  • packages/cli/src/managed-python-runtime.ts, packages/cli/src/runtime.ts, and packages/cli/src/commands/runtime-commands.ts.
  • packages/cli/src/managed-python-command.ts and ktx sl query runtime install policy flags.
  • packages/cli/src/managed-python-daemon.ts, daemon state files, and ktx runtime start / ktx runtime stop.
  • packages/cli/src/managed-local-embeddings.ts and setup embedding wiring.
  • scripts/build-public-npm-package.mjs, release-policy.json, and release smoke coverage for @kaelio/ktx.
  • scripts/package-artifacts.mjs release smoke coverage for lazy core runtime install, ktx sl query, runtime status, doctor, daemon start, daemon reuse, and daemon stop.
  • scripts/local-embeddings-runtime-smoke.mjs opt-in release smoke coverage for local-embeddings.

The next remaining semantic compute gap is that these CLI paths still create a raw Python semantic-layer compute port:

  • packages/cli/src/agent-runtime.ts
  • packages/cli/src/serve.ts

Those paths can call semantic-query, semantic-validate, and semantic-generate-sources through @ktx/context/daemon, so they must resolve the managed runtime just like ktx sl query.

This plan intentionally does not change live-database introspection or Looker table-identifier parsing. Those use daemon HTTP endpoints through local ingest adapters and fit a separate managed-daemon adapter plan.

File structure

  • Modify packages/cli/src/managed-python-command.ts: export a shared runtimeInstallPolicyFromFlags() helper so CLI commands do not duplicate --yes / --no-input behavior.
  • Modify packages/cli/src/managed-python-command.test.ts: cover the shared policy helper.
  • Modify packages/cli/src/commands/sl-commands.ts: replace its private runtime policy helper with the shared helper.
  • Modify packages/cli/src/agent-runtime.ts: create managed semantic compute when agent SL query needs Python and no dependency override is injected.
  • Modify packages/cli/src/agent-runtime.test.ts: cover the managed agent runtime path.
  • Modify packages/cli/src/agent.ts: pass CLI version, install policy, and CLI IO into default agent runtime creation for sl-query.
  • Modify packages/cli/src/agent.test.ts: cover runtime options passed through agent SL query execution.
  • Modify packages/cli/src/commands/agent-commands.ts: add --yes and --no-input to hidden ktx agent sl query.
  • Modify packages/cli/src/serve.ts: create managed semantic compute for ktx serve --semantic-compute when no explicit HTTP URL is provided.
  • Modify packages/cli/src/serve.test.ts: cover the managed MCP semantic compute path.
  • Modify packages/cli/src/commands/serve-commands.ts: add --yes and --no-input to ktx serve.
  • Modify packages/cli/src/index.test.ts: update CLI argument routing for the new managed runtime policy fields.

Task 1: Share managed runtime install policy parsing

Files:

  • Modify: packages/cli/src/managed-python-command.test.ts

  • Modify: packages/cli/src/managed-python-command.ts

  • Modify: packages/cli/src/commands/sl-commands.ts

  • Test: packages/cli/src/managed-python-command.test.ts

  • Test: packages/cli/src/index.test.ts

  • Step 1: Write failing policy helper tests

In packages/cli/src/managed-python-command.test.ts, update the import from ./managed-python-command.js to include runtimeInstallPolicyFromFlags:

import {
  createManagedPythonSemanticLayerComputePort,
  managedRuntimeInstallCommand,
  runtimeInstallPolicyFromFlags,
} from './managed-python-command.js';

Add this block after the existing describe('managedRuntimeInstallCommand', ...) block:

describe('runtimeInstallPolicyFromFlags', () => {
  it('maps command flags to managed runtime install policies', () => {
    expect(runtimeInstallPolicyFromFlags({})).toBe('prompt');
    expect(runtimeInstallPolicyFromFlags({ yes: false })).toBe('prompt');
    expect(runtimeInstallPolicyFromFlags({ yes: true })).toBe('auto');
    expect(runtimeInstallPolicyFromFlags({ input: false })).toBe('never');
  });

  it('rejects conflicting runtime install flags', () => {
    expect(() => runtimeInstallPolicyFromFlags({ yes: true, input: false })).toThrow(
      'Choose only one runtime install mode: --yes or --no-input',
    );
  });
});
  • Step 2: Run the failing helper tests

Run:

pnpm --filter @ktx/cli run test -- src/managed-python-command.test.ts

Expected: FAIL with an import error for runtimeInstallPolicyFromFlags.

  • Step 3: Export the shared policy helper

In packages/cli/src/managed-python-command.ts, add this function immediately after the KtxManagedPythonInstallPolicy type:

export function runtimeInstallPolicyFromFlags(options: {
  yes?: boolean;
  input?: boolean;
}): KtxManagedPythonInstallPolicy {
  if (options.yes === true && options.input === false) {
    throw new Error('Choose only one runtime install mode: --yes or --no-input');
  }
  if (options.yes === true) {
    return 'auto';
  }
  return options.input === false ? 'never' : 'prompt';
}
  • Step 4: Replace the private SL policy helper

In packages/cli/src/commands/sl-commands.ts, replace this import:

import type { KtxManagedPythonInstallPolicy } from '../managed-python-command.js';

with this import:

import { runtimeInstallPolicyFromFlags } from '../managed-python-command.js';

Delete this private function from packages/cli/src/commands/sl-commands.ts:

function runtimeInstallPolicy(options: { yes?: boolean; input?: boolean }): KtxManagedPythonInstallPolicy {
  if (options.yes === true && options.input === false) {
    throw new Error('Choose only one runtime install mode: --yes or --no-input');
  }
  if (options.yes === true) {
    return 'auto';
  }
  return options.input === false ? 'never' : 'prompt';
}

In the sl.command('query') action, replace:

runtimeInstallPolicy: runtimeInstallPolicy(options),

with:

runtimeInstallPolicy: runtimeInstallPolicyFromFlags(options),
  • Step 5: Run focused helper and routing tests

Run:

pnpm --filter @ktx/cli run test -- src/managed-python-command.test.ts src/index.test.ts

Expected: PASS.

  • Step 6: Commit the shared helper
git add packages/cli/src/managed-python-command.ts packages/cli/src/managed-python-command.test.ts packages/cli/src/commands/sl-commands.ts
git commit -m "refactor: share managed runtime install policy parsing"

Task 2: Use managed semantic compute for hidden agent SL query

Files:

  • Modify: packages/cli/src/agent-runtime.test.ts

  • Modify: packages/cli/src/agent-runtime.ts

  • Modify: packages/cli/src/agent.test.ts

  • Modify: packages/cli/src/agent.ts

  • Modify: packages/cli/src/commands/agent-commands.ts

  • Modify: packages/cli/src/index.test.ts

  • Test: packages/cli/src/agent-runtime.test.ts

  • Test: packages/cli/src/agent.test.ts

  • Test: packages/cli/src/index.test.ts

  • Step 1: Add failing agent runtime tests

In packages/cli/src/agent-runtime.test.ts, add this test after constructs local context ports with semantic compute and query executor:

  it('creates managed semantic compute when no test override is injected', async () => {
    const project = {
      projectDir: tempDir,
      configPath: join(tempDir, 'ktx.yaml'),
      config: { project: 'revenue', connections: {} },
      coreConfig: {},
      git: {},
      fileStore: {},
    } as never;
    const ports = { semanticLayer: {} } as never;
    const semanticLayerCompute = { query: vi.fn(), validateSources: vi.fn(), generateSources: vi.fn() };
    const loadProject = vi.fn(async () => project);
    const createContextTools = vi.fn(() => ports);
    const createManagedSemanticLayerCompute = vi.fn(async () => semanticLayerCompute);
    const { io } = makeIo();

    await expect(
      createKtxAgentRuntime(
        {
          projectDir: tempDir,
          enableSemanticCompute: true,
          enableQueryExecution: false,
          cliVersion: '0.2.0',
          runtimeInstallPolicy: 'auto',
          io,
        },
        {
          loadProject,
          createContextTools,
          createManagedSemanticLayerCompute,
        },
      ),
    ).resolves.toMatchObject({ project, ports, semanticLayerCompute });

    expect(createManagedSemanticLayerCompute).toHaveBeenCalledWith({
      cliVersion: '0.2.0',
      installPolicy: 'auto',
      io,
    });
    expect(createContextTools).toHaveBeenCalledWith(project, {
      semanticLayerCompute,
    });
  });
  • Step 2: Add failing agent command/runtime tests

In packages/cli/src/agent.test.ts, update the existing executes SL queries from a JSON query file test so the sl-query args include the managed runtime fields:

        {
          command: 'sl-query',
          projectDir: tempDir,
          json: true,
          connectionId: 'warehouse',
          queryFile,
          execute: true,
          maxRows: 100,
          cliVersion: '0.2.0',
          runtimeInstallPolicy: 'never',
        },

Add this test immediately after executes SL queries from a JSON query file:

  it('passes managed runtime options into default SL query runtime creation', async () => {
    const queryFile = join(tempDir, 'sl-query.json');
    const io = makeIo();
    const createRuntime = vi.fn(async () => runtime());
    await writeFile(queryFile, '{"measures":["total_revenue"],"dimensions":[]}', 'utf-8');

    await expect(
      runKtxAgent(
        {
          command: 'sl-query',
          projectDir: tempDir,
          json: true,
          connectionId: 'warehouse',
          queryFile,
          execute: false,
          cliVersion: '0.2.0',
          runtimeInstallPolicy: 'auto',
        },
        io.io,
        { createRuntime },
      ),
    ).resolves.toBe(0);

    expect(createRuntime).toHaveBeenCalledWith({
      projectDir: tempDir,
      enableSemanticCompute: true,
      enableQueryExecution: false,
      cliVersion: '0.2.0',
      runtimeInstallPolicy: 'auto',
      io: io.io,
    });
  });
  • Step 3: Add failing CLI routing tests

In packages/cli/src/index.test.ts, update the existing dispatches full hidden agent commands without exposing agent in root help case for agent sl query so its expected args include:

          cliVersion: '0.0.0-private',
          runtimeInstallPolicy: 'prompt',

Add this test after that existing full hidden agent command test:

  it('routes hidden agent SL query managed runtime policies', async () => {
    const autoIo = makeIo();
    const neverIo = makeIo();
    const conflictIo = makeIo();
    const agent = vi.fn(async () => 0);

    await expect(
      runKtxCli(
        [
          '--project-dir',
          tempDir,
          'agent',
          'sl',
          'query',
          '--json',
          '--connection-id',
          'warehouse',
          '--query-file',
          '/tmp/query.json',
          '--yes',
        ],
        autoIo.io,
        { agent },
      ),
    ).resolves.toBe(0);

    await expect(
      runKtxCli(
        [
          '--project-dir',
          tempDir,
          'agent',
          'sl',
          'query',
          '--json',
          '--connection-id',
          'warehouse',
          '--query-file',
          '/tmp/query.json',
          '--no-input',
        ],
        neverIo.io,
        { agent },
      ),
    ).resolves.toBe(0);

    await expect(
      runKtxCli(
        [
          '--project-dir',
          tempDir,
          'agent',
          'sl',
          'query',
          '--json',
          '--connection-id',
          'warehouse',
          '--query-file',
          '/tmp/query.json',
          '--yes',
          '--no-input',
        ],
        conflictIo.io,
        { agent },
      ),
    ).resolves.toBe(1);

    expect(agent).toHaveBeenNthCalledWith(
      1,
      {
        command: 'sl-query',
        projectDir: tempDir,
        json: true,
        connectionId: 'warehouse',
        queryFile: '/tmp/query.json',
        execute: false,
        cliVersion: '0.0.0-private',
        runtimeInstallPolicy: 'auto',
      },
      autoIo.io,
    );
    expect(agent).toHaveBeenNthCalledWith(
      2,
      {
        command: 'sl-query',
        projectDir: tempDir,
        json: true,
        connectionId: 'warehouse',
        queryFile: '/tmp/query.json',
        execute: false,
        cliVersion: '0.0.0-private',
        runtimeInstallPolicy: 'never',
      },
      neverIo.io,
    );
    expect(conflictIo.stderr()).toContain('Choose only one runtime install mode: --yes or --no-input');
  });
  • Step 4: Run the failing agent tests

Run:

pnpm --filter @ktx/cli run test -- src/agent-runtime.test.ts src/agent.test.ts src/index.test.ts

Expected: FAIL with TypeScript or runtime errors for createManagedSemanticLayerCompute, missing cliVersion, missing runtimeInstallPolicy, or unsupported hidden agent --yes / --no-input.

  • Step 5: Implement managed agent runtime creation

In packages/cli/src/agent-runtime.ts, replace the direct @ktx/context/daemon import:

import { createPythonSemanticLayerComputePort, type KtxSemanticLayerComputePort } from '@ktx/context/daemon';

with:

import type { KtxSemanticLayerComputePort } from '@ktx/context/daemon';
import {
  createManagedPythonSemanticLayerComputePort,
  type KtxManagedPythonInstallPolicy,
} from './managed-python-command.js';

Update KtxAgentRuntimeOptions to:

export interface KtxAgentRuntimeOptions {
  projectDir: string;
  enableSemanticCompute: boolean;
  enableQueryExecution: boolean;
  cliVersion?: string;
  runtimeInstallPolicy?: KtxManagedPythonInstallPolicy;
  io?: KtxCliIo;
}

Update KtxAgentRuntimeDeps to:

export interface KtxAgentRuntimeDeps {
  loadProject?: typeof loadKtxProject;
  createContextTools?: typeof createLocalProjectMcpContextPorts;
  createSemanticLayerCompute?: () => KtxSemanticLayerComputePort;
  createManagedSemanticLayerCompute?: typeof createManagedPythonSemanticLayerComputePort;
  createQueryExecutor?: () => KtxSqlQueryExecutorPort;
}

Add this helper before createKtxAgentRuntime:

async function createAgentSemanticLayerCompute(
  options: KtxAgentRuntimeOptions,
  deps: KtxAgentRuntimeDeps,
): Promise<KtxSemanticLayerComputePort | undefined> {
  if (!options.enableSemanticCompute) {
    return undefined;
  }
  if (deps.createSemanticLayerCompute) {
    return deps.createSemanticLayerCompute();
  }
  if (!options.cliVersion || !options.runtimeInstallPolicy || !options.io) {
    throw new Error('Managed Python semantic compute requires cliVersion, runtimeInstallPolicy, and io.');
  }
  const createManagedSemanticLayerCompute =
    deps.createManagedSemanticLayerCompute ?? createManagedPythonSemanticLayerComputePort;
  return createManagedSemanticLayerCompute({
    cliVersion: options.cliVersion,
    installPolicy: options.runtimeInstallPolicy,
    io: options.io,
  });
}

In createKtxAgentRuntime, replace:

  const semanticLayerCompute = options.enableSemanticCompute
    ? (deps.createSemanticLayerCompute ?? createPythonSemanticLayerComputePort)()
    : undefined;

with:

  const semanticLayerCompute = await createAgentSemanticLayerCompute(options, deps);
  • Step 6: Pass runtime options through agent execution

In packages/cli/src/agent.ts, add this import:

import type { KtxManagedPythonInstallPolicy } from './managed-python-command.js';

Update the sl-query variant in KtxAgentArgs to:

  | {
      command: 'sl-query';
      projectDir: string;
      json: true;
      connectionId: string;
      queryFile: string;
      execute: boolean;
      maxRows?: number;
      cliVersion: string;
      runtimeInstallPolicy: KtxManagedPythonInstallPolicy;
    }

Update KtxAgentDeps.createRuntime to use the shared runtime options type:

  createRuntime?: (options: {
    projectDir: string;
    enableSemanticCompute: boolean;
    enableQueryExecution: boolean;
    cliVersion?: string;
    runtimeInstallPolicy?: KtxManagedPythonInstallPolicy;
    io?: KtxCliIo;
  }) => Promise<KtxAgentRuntime>;

Change runtimeFor from:

async function runtimeFor(args: KtxAgentArgs, deps: KtxAgentDeps): Promise<KtxAgentRuntime> {
  const needsSemanticCompute = args.command === 'sl-query';
  const needsQueryExecution = args.command === 'sql-execute' || (args.command === 'sl-query' && args.execute);
  return deps.createRuntime
    ? deps.createRuntime({
        projectDir: args.projectDir,
        enableSemanticCompute: needsSemanticCompute,
        enableQueryExecution: needsQueryExecution,
      })
    : createKtxAgentRuntime(
        {
          projectDir: args.projectDir,
          enableSemanticCompute: needsSemanticCompute,
          enableQueryExecution: needsQueryExecution,
        },
        deps,
      );
}

to:

async function runtimeFor(args: KtxAgentArgs, deps: KtxAgentDeps, io: KtxCliIo): Promise<KtxAgentRuntime> {
  const needsSemanticCompute = args.command === 'sl-query';
  const needsQueryExecution = args.command === 'sql-execute' || (args.command === 'sl-query' && args.execute);
  const runtimeOptions = {
    projectDir: args.projectDir,
    enableSemanticCompute: needsSemanticCompute,
    enableQueryExecution: needsQueryExecution,
    ...(args.command === 'sl-query'
      ? {
          cliVersion: args.cliVersion,
          runtimeInstallPolicy: args.runtimeInstallPolicy,
          io,
        }
      : {}),
  };
  return deps.createRuntime ? deps.createRuntime(runtimeOptions) : createKtxAgentRuntime(runtimeOptions, deps);
}

In runKtxAgent, replace:

    const runtime = await runtimeFor(args, deps);

with:

    const runtime = await runtimeFor(args, deps, io);
  • Step 7: Add hidden agent runtime policy flags

In packages/cli/src/commands/agent-commands.ts, add this import:

import { runtimeInstallPolicyFromFlags } from '../managed-python-command.js';

In the agent sl query command chain, add these options after .option('--execute', ...):

    .option('--yes', 'Install the managed Python runtime without prompting when required', false)
    .option('--no-input', 'Disable interactive managed runtime installation')

Update the action options type from:

        options: { connectionId: string; queryFile: string; execute: boolean; maxRows?: number },

to:

        options: {
          connectionId: string;
          queryFile: string;
          execute: boolean;
          maxRows?: number;
          yes?: boolean;
          input?: boolean;
        },

Add these fields to the runAgent argument object:

          cliVersion: context.packageInfo.version,
          runtimeInstallPolicy: runtimeInstallPolicyFromFlags(options),
  • Step 8: Run focused agent tests

Run:

pnpm --filter @ktx/cli run test -- src/agent-runtime.test.ts src/agent.test.ts src/index.test.ts

Expected: PASS.

  • Step 9: Commit the agent integration
git add packages/cli/src/agent-runtime.ts packages/cli/src/agent-runtime.test.ts packages/cli/src/agent.ts packages/cli/src/agent.test.ts packages/cli/src/commands/agent-commands.ts packages/cli/src/index.test.ts
git commit -m "feat: use managed runtime for agent semantic queries"

Task 3: Use managed semantic compute for MCP serve

Files:

  • Modify: packages/cli/src/serve.test.ts

  • Modify: packages/cli/src/serve.ts

  • Modify: packages/cli/src/commands/serve-commands.ts

  • Modify: packages/cli/src/index.test.ts

  • Test: packages/cli/src/serve.test.ts

  • Test: packages/cli/src/index.test.ts

  • Step 1: Add a failing serve managed runtime test

In packages/cli/src/serve.test.ts, add this helper after the imports:

function makeManagedRuntimeIo() {
  let stdout = '';
  let stderr = '';
  return {
    io: {
      stdout: { write: (chunk: string) => (stdout += chunk) },
      stderr: { write: (chunk: string) => (stderr += chunk) },
    },
    stdout: () => stdout,
    stderr: () => stderr,
  };
}

Add this test before uses the HTTP semantic compute port when a daemon URL is provided:

  it('uses managed semantic compute when MCP semantic compute has no explicit HTTP URL', async () => {
    const project = { projectDir: '/tmp/ktx-project', config: { connections: {} } } as never;
    const semanticLayerCompute = { query: vi.fn(), validateSources: vi.fn(), generateSources: vi.fn() };
    const createManagedSemanticLayerCompute = vi.fn(async () => semanticLayerCompute);
    const createContextTools = vi.fn(() => ({ connections: { list: async () => [] } }));
    const managedRuntimeIo = makeManagedRuntimeIo();

    await expect(
      runKtxServeStdio(
        {
          mcp: 'stdio',
          projectDir: '/tmp/ktx-project',
          userId: 'agent',
          semanticCompute: true,
          semanticComputeUrl: undefined,
          databaseIntrospectionUrl: undefined,
          executeQueries: false,
          memoryCapture: false,
          memoryModel: undefined,
          cliVersion: '0.2.0',
          runtimeInstallPolicy: 'auto',
        },
        {
          loadProject: async () => project,
          createContextTools,
          createManagedSemanticLayerCompute,
          managedRuntimeIo: managedRuntimeIo.io,
          createServer: vi.fn(() => ({ connect: vi.fn(async () => undefined) }) as never),
          createTransport: vi.fn(() => ({}) as never),
          stderr: { write: vi.fn() },
        },
      ),
    ).resolves.toBe(0);

    expect(createManagedSemanticLayerCompute).toHaveBeenCalledWith({
      cliVersion: '0.2.0',
      installPolicy: 'auto',
      io: managedRuntimeIo.io,
    });
    expect(createContextTools).toHaveBeenCalledWith(
      project,
      expect.objectContaining({
        semanticLayerCompute,
      }),
    );
  });
  • Step 2: Add failing serve routing tests

In packages/cli/src/index.test.ts, update both existing serveStdio expectations so the expected args include:

      cliVersion: '0.0.0-private',
      runtimeInstallPolicy: 'prompt',

Add this test after dispatches serve public command options through Commander:

  it('routes serve managed runtime install policies', async () => {
    const autoIo = makeIo();
    const neverIo = makeIo();
    const conflictIo = makeIo();
    const serveStdio = vi.fn(async () => 0);

    await expect(
      runKtxCli(['serve', '--mcp', 'stdio', '--project-dir', tempDir, '--semantic-compute', '--yes'], autoIo.io, {
        serveStdio,
      }),
    ).resolves.toBe(0);
    await expect(
      runKtxCli(['serve', '--mcp', 'stdio', '--project-dir', tempDir, '--semantic-compute', '--no-input'], neverIo.io, {
        serveStdio,
      }),
    ).resolves.toBe(0);
    await expect(
      runKtxCli(
        ['serve', '--mcp', 'stdio', '--project-dir', tempDir, '--semantic-compute', '--yes', '--no-input'],
        conflictIo.io,
        { serveStdio },
      ),
    ).resolves.toBe(1);

    expect(serveStdio).toHaveBeenNthCalledWith(1, {
      mcp: 'stdio',
      projectDir: tempDir,
      userId: 'local',
      semanticCompute: true,
      semanticComputeUrl: undefined,
      databaseIntrospectionUrl: undefined,
      executeQueries: false,
      memoryCapture: false,
      memoryModel: undefined,
      cliVersion: '0.0.0-private',
      runtimeInstallPolicy: 'auto',
    });
    expect(serveStdio).toHaveBeenNthCalledWith(2, {
      mcp: 'stdio',
      projectDir: tempDir,
      userId: 'local',
      semanticCompute: true,
      semanticComputeUrl: undefined,
      databaseIntrospectionUrl: undefined,
      executeQueries: false,
      memoryCapture: false,
      memoryModel: undefined,
      cliVersion: '0.0.0-private',
      runtimeInstallPolicy: 'never',
    });
    expect(conflictIo.stderr()).toContain('Choose only one runtime install mode: --yes or --no-input');
  });
  • Step 3: Run the failing serve tests

Run:

pnpm --filter @ktx/cli run test -- src/serve.test.ts src/index.test.ts

Expected: FAIL with missing createManagedSemanticLayerCompute support and missing cliVersion / runtimeInstallPolicy fields in command routing.

  • Step 4: Implement managed serve semantic compute

In packages/cli/src/serve.ts, add this import:

import type { KtxCliIo } from './cli-runtime.js';
import {
  createManagedPythonSemanticLayerComputePort,
  type KtxManagedPythonInstallPolicy,
} from './managed-python-command.js';

Update KtxServeArgs to:

export interface KtxServeArgs {
  mcp: 'stdio';
  projectDir: string;
  userId: string;
  semanticCompute: boolean;
  semanticComputeUrl?: string;
  databaseIntrospectionUrl?: string;
  executeQueries: boolean;
  memoryCapture: boolean;
  memoryModel?: string;
  cliVersion?: string;
  runtimeInstallPolicy?: KtxManagedPythonInstallPolicy;
}

Update KtxServeDeps to include:

  createManagedSemanticLayerCompute?: typeof createManagedPythonSemanticLayerComputePort;
  managedRuntimeIo?: KtxCliIo;

Add these helpers before runKtxServeStdio:

function requiredManagedRuntimeCliVersion(args: KtxServeArgs): string {
  if (!args.cliVersion) {
    throw new Error('Managed Python semantic compute requires a CLI version.');
  }
  return args.cliVersion;
}

async function createServeSemanticLayerCompute(
  args: KtxServeArgs,
  deps: KtxServeDeps,
): Promise<KtxSemanticLayerComputePort | undefined> {
  if (!args.semanticCompute) {
    return undefined;
  }
  if (args.semanticComputeUrl) {
    return (deps.createHttpSemanticLayerCompute ?? ((baseUrl) => createHttpSemanticLayerComputePort({ baseUrl })))(
      args.semanticComputeUrl,
    );
  }
  if (deps.createSemanticLayerCompute) {
    return deps.createSemanticLayerCompute();
  }
  const createManagedSemanticLayerCompute =
    deps.createManagedSemanticLayerCompute ?? createManagedPythonSemanticLayerComputePort;
  return createManagedSemanticLayerCompute({
    cliVersion: requiredManagedRuntimeCliVersion(args),
    installPolicy: args.runtimeInstallPolicy ?? 'prompt',
    io: deps.managedRuntimeIo ?? process,
  });
}

In runKtxServeStdio, replace:

  const semanticLayerCompute = args.semanticCompute
    ? args.semanticComputeUrl
      ? (deps.createHttpSemanticLayerCompute ?? ((baseUrl) => createHttpSemanticLayerComputePort({ baseUrl })))(
          args.semanticComputeUrl,
        )
      : (deps.createSemanticLayerCompute ?? createPythonSemanticLayerComputePort)()
    : undefined;

with:

  const semanticLayerCompute = await createServeSemanticLayerCompute(args, deps);

Remove createPythonSemanticLayerComputePort from the @ktx/context/daemon import list.

  • Step 5: Add serve runtime policy flags

In packages/cli/src/commands/serve-commands.ts, add this import:

import { runtimeInstallPolicyFromFlags } from '../managed-python-command.js';

Add these command options after .option('--semantic-compute-url <url>', ...):

    .option('--yes', 'Install the managed Python runtime without prompting when required', false)
    .option('--no-input', 'Disable interactive managed runtime installation')

Add these fields to the KtxServeArgs object:

        cliVersion: context.packageInfo.version,
        runtimeInstallPolicy: runtimeInstallPolicyFromFlags(options),
  • Step 6: Run focused serve tests

Run:

pnpm --filter @ktx/cli run test -- src/serve.test.ts src/index.test.ts

Expected: PASS.

  • Step 7: Commit the serve integration
git add packages/cli/src/serve.ts packages/cli/src/serve.test.ts packages/cli/src/commands/serve-commands.ts packages/cli/src/index.test.ts
git commit -m "feat: use managed runtime for MCP semantic compute"

Task 4: Verify managed semantic runtime integration

Files:

  • Verify: packages/cli/src/managed-python-command.ts

  • Verify: packages/cli/src/agent-runtime.ts

  • Verify: packages/cli/src/agent.ts

  • Verify: packages/cli/src/commands/agent-commands.ts

  • Verify: packages/cli/src/serve.ts

  • Verify: packages/cli/src/commands/serve-commands.ts

  • Verify: packages/cli/src/index.test.ts

  • Step 1: Run all focused CLI tests touched by this plan

Run:

pnpm --filter @ktx/cli run test -- src/managed-python-command.test.ts src/agent-runtime.test.ts src/agent.test.ts src/serve.test.ts src/index.test.ts

Expected: PASS.

  • Step 2: Run CLI type-check

Run:

pnpm --filter @ktx/cli run type-check

Expected: PASS.

  • Step 3: Run CLI tests

Run:

pnpm --filter @ktx/cli run test

Expected: PASS.

  • Step 4: Run package build

Run:

pnpm --filter @ktx/cli run build

Expected: PASS.

  • Step 5: Commit any verification fixes

If verification required code edits, run:

git add packages/cli/src/managed-python-command.ts packages/cli/src/managed-python-command.test.ts packages/cli/src/commands/sl-commands.ts packages/cli/src/agent-runtime.ts packages/cli/src/agent-runtime.test.ts packages/cli/src/agent.ts packages/cli/src/agent.test.ts packages/cli/src/commands/agent-commands.ts packages/cli/src/serve.ts packages/cli/src/serve.test.ts packages/cli/src/commands/serve-commands.ts packages/cli/src/index.test.ts
git commit -m "fix: verify managed semantic runtime surfaces"

If no files changed after Step 1 through Step 4, do not create an empty commit.

Acceptance criteria

  • ktx agent sl query has --yes and --no-input managed runtime policy flags.
  • ktx agent sl query --yes passes runtimeInstallPolicy: 'auto' and the current CLI package version into default runtime creation.
  • ktx agent sl query --no-input passes runtimeInstallPolicy: 'never'.
  • ktx agent sl query --yes --no-input exits with Choose only one runtime install mode: --yes or --no-input.
  • Default agent SL query runtime creation uses createManagedPythonSemanticLayerComputePort() and therefore invokes the bundled managed ktx-daemon executable.
  • ktx serve --mcp stdio --semantic-compute has --yes and --no-input managed runtime policy flags.
  • ktx serve --mcp stdio --semantic-compute --yes passes runtimeInstallPolicy: 'auto' and the current CLI package version into serve runtime creation.
  • ktx serve --mcp stdio --semantic-compute --no-input passes runtimeInstallPolicy: 'never'.
  • ktx serve --mcp stdio --semantic-compute-url <url> continues to use the explicit HTTP semantic compute port and does not install or start a managed runtime.
  • Focused CLI tests, full CLI tests, CLI type-check, and CLI build pass.

Self-review

  • Spec coverage: this plan extends managed Python one-shot semantic compute to hidden agent SL query and MCP serve --semantic-compute, covering additional semantic query, validation, and source-generation paths that use @ktx/context/daemon.
  • Remaining intentional gap: local ingest daemon-backed database introspection and Looker SQL table-identifier parsing still need a managed daemon adapter plan because they use HTTP daemon endpoints rather than the one-shot semantic compute port.
  • Placeholder scan: all steps contain concrete edits, commands, and expected results.
  • Type consistency: runtime policy values stay prompt, auto, and never; runtime feature values stay core and local-embeddings; package version fields are named cliVersion.