# Bundled Python Runtime Wheel Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use > superpowers:subagent-driven-development (recommended) or > superpowers:executing-plans to implement this plan task-by-task. Steps use > checkbox (`- [ ]`) syntax for tracking. **Goal:** Build and package one bundled `kaelio-ktx` Python wheel that contains KTX-owned Python runtime code and keeps local embedding dependencies optional. **Architecture:** Add a deterministic Node assembly script that copies the existing `semantic_layer` and `ktx_daemon` source trees into a temporary wheel source tree, writes a runtime-only `pyproject.toml`, and builds one wheel with `uv build`. Wire package artifacts so the CLI npm tarball includes the bundled wheel plus a checksum manifest under `assets/python/`. **Tech Stack:** Node 22 ESM scripts, `node:test`, `uv`, Hatchling, Python 3.13, pnpm, TypeScript package artifacts. --- ## Existing status This plan is based on `docs/superpowers/specs/2026-05-11-npm-managed-python-runtime-design.md`. There are no committed plan files under `docs/superpowers/plans/` in this worktree or in git history for this spec. The spec itself is the only tracked Superpowers document. The following pieces are already implemented: - `packages/context/src/daemon/semantic-layer-compute.ts` can invoke `python -m ktx_daemon` for one-shot semantic-layer operations. - `python/ktx-daemon` exposes `ktx-daemon` one-shot commands and an HTTP `serve-http` daemon with `/health`. - `scripts/package-artifacts.mjs` builds npm package tarballs and separate `ktx-sl` and `ktx-daemon` Python artifacts. - `scripts/package-artifacts.mjs` writes a checksummed artifact manifest. The following spec requirements are not implemented yet: - A single public `@kaelio/ktx` npm surface. - One KTX-owned bundled Python wheel inside the npm package. - A managed runtime root, installer, runtime manifest, and runtime command family. - Lazy `local-embeddings` installation that keeps `sentence-transformers` and `torch` out of the default Python dependency set. This plan implements the bundled wheel prerequisite. Runtime install commands must be planned after this lands because they need a real wheel payload and checksum manifest to install. ## File structure - Create `scripts/build-python-runtime-wheel.mjs`: assembles the temporary runtime wheel source tree and runs `uv build`. - Create `scripts/build-python-runtime-wheel.test.mjs`: tests source copying, generated `pyproject.toml`, and the `uv build` command shape. - Modify `scripts/package-artifacts.mjs`: builds the runtime wheel before npm packing, copies it into `packages/cli/assets/python/`, includes it in the artifact manifest, and installs it in artifact smoke tests. - Modify `scripts/package-artifacts.test.mjs`: covers runtime wheel metadata, manifest entries, install arguments, and CLI asset copy behavior. - Modify `scripts/release-readiness.test.mjs`: expects `kaelio-ktx` in Python release metadata and policy fixtures. - Modify `release-policy.json`: lists `kaelio-ktx` as a CI-only Python artifact. - Modify `python/ktx-daemon/pyproject.toml`: moves `sentence-transformers` and `torch` to a `local-embeddings` optional dependency group. - Modify `uv.lock`: records the dependency metadata change. - Modify `.gitignore`: ignores generated `packages/cli/assets/python/` contents. ## Plan status No earlier plans were found for this spec. This is plan 1 for the spec. ### Task 1: Add failing tests for the runtime wheel builder **Files:** - Create: `scripts/build-python-runtime-wheel.test.mjs` - Test: `scripts/build-python-runtime-wheel.test.mjs` - [ ] **Step 1: Write the failing test file** Create `scripts/build-python-runtime-wheel.test.mjs` with this content: ```javascript import assert from 'node:assert/strict'; import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { describe, it } from 'node:test'; import { RUNTIME_WHEEL_DISTRIBUTION_NAME, RUNTIME_WHEEL_PACKAGE_VERSION, createRuntimeWheelBuildTree, runtimeWheelBuildCommand, runtimeWheelLayout, runtimeWheelPyproject, } from './build-python-runtime-wheel.mjs'; async function writeRuntimeSourceFixture(root) { await mkdir(join(root, 'python', 'ktx-sl', 'semantic_layer'), { recursive: true, }); await mkdir(join(root, 'python', 'ktx-daemon', 'src', 'ktx_daemon'), { recursive: true, }); await writeFile( join(root, 'python', 'ktx-sl', 'semantic_layer', '__init__.py'), 'SEMANTIC_LAYER_FIXTURE = True\n', ); await writeFile( join(root, 'python', 'ktx-daemon', 'src', 'ktx_daemon', '__init__.py'), 'KTX_DAEMON_FIXTURE = True\n', ); await writeFile( join(root, 'python', 'ktx-daemon', 'src', 'ktx_daemon', '__main__.py'), 'def main():\n return 0\n', ); } describe('runtimeWheelLayout', () => { it('uses stable source, build, and output paths', () => { const layout = runtimeWheelLayout('/repo/ktx'); assert.equal(layout.rootDir, '/repo/ktx'); assert.equal(layout.semanticLayerSourceDir, '/repo/ktx/python/ktx-sl/semantic_layer'); assert.equal(layout.daemonSourceDir, '/repo/ktx/python/ktx-daemon/src/ktx_daemon'); assert.equal(layout.buildRoot, '/repo/ktx/dist/runtime-wheel-src'); assert.equal(layout.outputDir, '/repo/ktx/dist/artifacts/python'); }); }); describe('runtimeWheelPyproject', () => { it('describes one kaelio-ktx wheel with lazy local embeddings', () => { const pyproject = runtimeWheelPyproject(); assert.match(pyproject, /name = "kaelio-ktx"/); assert.match(pyproject, /version = "0\.1\.0"/); assert.match(pyproject, /ktx-daemon = "ktx_daemon\.__main__:main"/); assert.match(pyproject, /packages = \["semantic_layer", "ktx_daemon"\]/); assert.match(pyproject, /\[project\.optional-dependencies\]/); assert.match(pyproject, /local-embeddings = \[/); assert.match(pyproject, /"sentence-transformers>=5\.1\.1"/); assert.match(pyproject, /"torch>=2\.2\.0"/); assert.doesNotMatch( pyproject.match(/dependencies = \[[\s\S]*?\]/)?.[0] ?? '', /sentence-transformers|torch/, ); }); }); describe('createRuntimeWheelBuildTree', () => { it('copies KTX-owned Python packages into the build tree', async () => { const root = await mkdtemp(join(tmpdir(), 'ktx-runtime-wheel-test-')); try { await writeRuntimeSourceFixture(root); const layout = runtimeWheelLayout(root); await createRuntimeWheelBuildTree(layout); assert.equal( await readFile(join(layout.buildRoot, 'semantic_layer', '__init__.py'), 'utf8'), 'SEMANTIC_LAYER_FIXTURE = True\n', ); assert.equal( await readFile(join(layout.buildRoot, 'ktx_daemon', '__main__.py'), 'utf8'), 'def main():\n return 0\n', ); const pyproject = await readFile(join(layout.buildRoot, 'pyproject.toml'), 'utf8'); assert.match(pyproject, /name = "kaelio-ktx"/); assert.match(pyproject, /local-embeddings = \[/); const readme = await readFile(join(layout.buildRoot, 'README.md'), 'utf8'); assert.match(readme, /Bundled Python runtime wheel for KTX/); } finally { await rm(root, { recursive: true, force: true }); } }); }); describe('runtimeWheelBuildCommand', () => { it('runs uv build against the generated build tree', () => { const layout = runtimeWheelLayout('/repo/ktx'); assert.deepEqual(runtimeWheelBuildCommand(layout), { command: 'uv', args: [ 'build', '--wheel', '--out-dir', '/repo/ktx/dist/artifacts/python', '/repo/ktx/dist/runtime-wheel-src', ], cwd: '/repo/ktx', }); assert.equal(RUNTIME_WHEEL_DISTRIBUTION_NAME, 'kaelio-ktx'); assert.equal(RUNTIME_WHEEL_PACKAGE_VERSION, '0.1.0'); }); }); ``` - [ ] **Step 2: Run the failing test** Run: ```bash node --test scripts/build-python-runtime-wheel.test.mjs ``` Expected: FAIL with an import error for `./build-python-runtime-wheel.mjs`. ### Task 2: Implement the runtime wheel builder **Files:** - Create: `scripts/build-python-runtime-wheel.mjs` - Test: `scripts/build-python-runtime-wheel.test.mjs` - [ ] **Step 1: Create the builder script** Create `scripts/build-python-runtime-wheel.mjs` with this content: ```javascript #!/usr/bin/env node import { execFile } from 'node:child_process'; import { cp, mkdir, readFile, rm, writeFile } from 'node:fs/promises'; import { dirname, join, resolve } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; import { promisify } from 'node:util'; const execFileAsync = promisify(execFile); export const RUNTIME_WHEEL_DISTRIBUTION_NAME = 'kaelio-ktx'; export const RUNTIME_WHEEL_NORMALIZED_NAME = 'kaelio_ktx'; export const RUNTIME_WHEEL_PACKAGE_VERSION = '0.1.0'; function scriptRootDir() { return resolve(dirname(fileURLToPath(import.meta.url)), '..'); } export function runtimeWheelLayout(rootDir = scriptRootDir()) { return { rootDir, semanticLayerSourceDir: join(rootDir, 'python', 'ktx-sl', 'semantic_layer'), daemonSourceDir: join(rootDir, 'python', 'ktx-daemon', 'src', 'ktx_daemon'), buildRoot: join(rootDir, 'dist', 'runtime-wheel-src'), outputDir: join(rootDir, 'dist', 'artifacts', 'python'), }; } export function runtimeWheelPyproject() { return `[project] name = "${RUNTIME_WHEEL_DISTRIBUTION_NAME}" version = "${RUNTIME_WHEEL_PACKAGE_VERSION}" description = "Bundled Python runtime payload for the KTX npm package" readme = "README.md" requires-python = ">=3.13" license = "Apache-2.0" dependencies = [ "fastapi>=0.115.0", "lkml>=1.3.7", "numpy>=2.2.6", "orjson>=3.11.4", "pandas>=2.2.3", "psycopg[binary]>=3.2.0", "pydantic>=2.9.0", "pyyaml>=6", "requests>=2.32.0", "sqlglot>=26", "uvicorn[standard]>=0.32.0", ] [project.optional-dependencies] local-embeddings = [ "sentence-transformers>=5.1.1", "torch>=2.2.0", ] [project.scripts] ktx-daemon = "ktx_daemon.__main__:main" [project.urls] Homepage = "https://github.com/kaelio/ktx" Repository = "https://github.com/kaelio/ktx" Issues = "https://github.com/kaelio/ktx/issues" [build-system] requires = ["hatchling"] build-backend = "hatchling.build" [tool.hatch.build.targets.wheel] packages = ["semantic_layer", "ktx_daemon"] `; } export function runtimeWheelReadme() { return `# kaelio-ktx Python runtime Bundled Python runtime wheel for KTX. This wheel is built from the repository's \`semantic_layer\` and \`ktx_daemon\` source trees for inclusion in the npm package. It is not a separate public PyPI release artifact. `; } export async function createRuntimeWheelBuildTree(layout = runtimeWheelLayout()) { await rm(layout.buildRoot, { recursive: true, force: true }); await mkdir(layout.buildRoot, { recursive: true }); await cp(layout.semanticLayerSourceDir, join(layout.buildRoot, 'semantic_layer'), { recursive: true, }); await cp(layout.daemonSourceDir, join(layout.buildRoot, 'ktx_daemon'), { recursive: true, }); await writeFile(join(layout.buildRoot, 'pyproject.toml'), runtimeWheelPyproject()); await writeFile(join(layout.buildRoot, 'README.md'), runtimeWheelReadme()); } export function runtimeWheelBuildCommand(layout = runtimeWheelLayout()) { return { command: 'uv', args: ['build', '--wheel', '--out-dir', layout.outputDir, layout.buildRoot], cwd: layout.rootDir, }; } async function runCommand(command, args, options) { const result = await execFileAsync(command, args, { cwd: options.cwd, encoding: 'utf8', maxBuffer: 1024 * 1024 * 20, }); if (result.stdout) { process.stdout.write(result.stdout); } if (result.stderr) { process.stderr.write(result.stderr); } } export async function buildRuntimeWheel(layout = runtimeWheelLayout()) { await mkdir(layout.outputDir, { recursive: true }); await createRuntimeWheelBuildTree(layout); const command = runtimeWheelBuildCommand(layout); await runCommand(command.command, command.args, { cwd: command.cwd }); const pyproject = await readFile(join(layout.buildRoot, 'pyproject.toml'), 'utf8'); return { buildRoot: layout.buildRoot, outputDir: layout.outputDir, pyproject, }; } async function main() { await buildRuntimeWheel(runtimeWheelLayout()); } if (import.meta.url === pathToFileURL(process.argv[1] ?? '').href) { try { await main(); } catch (error) { process.stderr.write(`${error instanceof Error ? error.stack : String(error)}\n`); process.exitCode = 1; } } ``` - [ ] **Step 2: Run the builder test** Run: ```bash node --test scripts/build-python-runtime-wheel.test.mjs ``` Expected: PASS. - [ ] **Step 3: Commit the builder** Run: ```bash git add scripts/build-python-runtime-wheel.mjs scripts/build-python-runtime-wheel.test.mjs git commit -m "build: add bundled python runtime wheel builder" ``` ### Task 3: Move heavy local embedding dependencies behind an extra **Files:** - Modify: `python/ktx-daemon/pyproject.toml` - Modify: `uv.lock` - Test: `python/ktx-daemon/tests/test_embeddings.py` - Test: `scripts/build-python-runtime-wheel.test.mjs` - [ ] **Step 1: Update daemon dependencies** In `python/ktx-daemon/pyproject.toml`, remove these two lines from `[project].dependencies`: ```toml "sentence-transformers>=5.1.1", "torch>=2.2.0", ``` Add this block immediately after `[project.scripts]`: ```toml [project.optional-dependencies] local-embeddings = [ "sentence-transformers>=5.1.1", "torch>=2.2.0", ] ``` The relevant section must read: ```toml [project] name = "ktx-daemon" version = "0.1.0" description = "Portable compute package for KTX semantic-layer operations" readme = "README.md" requires-python = ">=3.13" license = "Apache-2.0" dependencies = [ "fastapi>=0.115.0", "ktx-sl", "lkml>=1.3.7", "numpy>=2.2.6", "orjson>=3.11.4", "pandas>=2.2.3", "psycopg[binary]>=3.2.0", "pydantic>=2.9.0", "requests>=2.32.0", "sqlglot>=26", "uvicorn[standard]>=0.32.0", ] [project.scripts] ktx-daemon = "ktx_daemon.__main__:main" [project.optional-dependencies] local-embeddings = [ "sentence-transformers>=5.1.1", "torch>=2.2.0", ] ``` - [ ] **Step 2: Refresh the uv lockfile** Run: ```bash uv lock ``` Expected: PASS and `uv.lock` records the `ktx-daemon` optional dependency metadata. If the local `uv` version is older than `tool.uv.required-version`, record the version mismatch and do not edit `pyproject.toml` to lower the pin. - [ ] **Step 3: Run Python tests that cover lazy embedding imports** Run: ```bash uv run pytest python/ktx-daemon/tests/test_embeddings.py -q ``` Expected: PASS. The tests use injected fake providers and do not require `sentence-transformers` or `torch`. - [ ] **Step 4: Run the runtime wheel metadata test** Run: ```bash node --test scripts/build-python-runtime-wheel.test.mjs ``` Expected: PASS and the generated runtime `pyproject.toml` keeps `sentence-transformers` and `torch` under `local-embeddings`. - [ ] **Step 5: Commit the dependency split** Run: ```bash git add python/ktx-daemon/pyproject.toml uv.lock git commit -m "build: make local embedding dependencies optional" ``` ### Task 4: Add artifact tests for the bundled runtime wheel **Files:** - Modify: `scripts/package-artifacts.test.mjs` - Test: `scripts/package-artifacts.test.mjs` - [ ] **Step 1: Extend imports** In `scripts/package-artifacts.test.mjs`, extend the import from `./package-artifacts.mjs` with these names: ```javascript CLI_PYTHON_ASSET_MANIFEST, RUNTIME_WHEEL_DISTRIBUTION_NAME, RUNTIME_WHEEL_NORMALIZED_NAME, RUNTIME_WHEEL_PACKAGE_VERSION, copyRuntimeWheelAssets, ``` - [ ] **Step 2: Update Python metadata fixtures** In `writeReleaseMetadataInputs`, keep the existing `ktx-sl` and `ktx-daemon` fixture files and add no new on-disk Python package. The runtime wheel metadata will come from constants exported by `package-artifacts.mjs`. - [ ] **Step 3: Update uploadable artifact fixtures** In `writeUploadableArtifactFixtures`, add this runtime wheel entry to `fileContents`: ```javascript [ join(layout.pythonDir, 'kaelio_ktx-0.1.0-py3-none-any.whl'), 'kaelio-ktx-runtime-wheel', ], ``` - [ ] **Step 4: Update build command expectations** Replace the `buildArtifactCommands` expectations with these three assertions: ```javascript assert.deepEqual( commands.slice(0, NPM_ARTIFACT_PACKAGES.length).map((command) => [command.command, command.args]), NPM_ARTIFACT_PACKAGES.map((packageInfo) => ['pnpm', ['--filter', packageInfo.name, 'run', 'build']]), ); assert.deepEqual( commands .slice(NPM_ARTIFACT_PACKAGES.length, NPM_ARTIFACT_PACKAGES.length + 3) .map((command) => [command.command, command.args]), [ [ process.execPath, ['scripts/build-python-runtime-wheel.mjs'], ], [ 'uv', ['build', '--package', 'ktx-sl', '--out-dir', '/repo/ktx/dist/artifacts/python'], ], [ 'uv', ['build', '--package', 'ktx-daemon', '--out-dir', '/repo/ktx/dist/artifacts/python'], ], ], ); assert.deepEqual( commands.slice(NPM_ARTIFACT_PACKAGES.length + 3).map((command) => [command.command, command.args]), NPM_ARTIFACT_PACKAGES.map((packageInfo) => [ 'pnpm', ['--filter', packageInfo.name, 'pack', '--out', layout.npmTarballs[packageInfo.name]], ]), ); ``` - [ ] **Step 5: Update release metadata expectations** In the `packageReleaseMetadata` test, add this Python metadata entry after `ktx-daemon`: ```javascript { ecosystem: 'python', packageName: 'kaelio-ktx', packageRoot: 'python/runtime-wheel', packageVersion: '0.1.0', private: false, releaseMode: 'ci-artifact-only', }, ``` - [ ] **Step 6: Update Python artifact discovery expectations** In the `findPythonArtifacts` test, create the runtime wheel fixture: ```javascript await writeFile(join(root, 'kaelio_ktx-0.1.0-py3-none-any.whl'), ''); ``` Then update the expected object: ```javascript assert.deepEqual(await findPythonArtifacts(root), { runtimeWheel: join(root, 'kaelio_ktx-0.1.0-py3-none-any.whl'), ktxSlWheel: join(root, 'ktx_sl-0.1.0-py3-none-any.whl'), ktxSlSdist: join(root, 'ktx_sl-0.1.0.tar.gz'), ktxDaemonWheel: join(root, 'ktx_daemon-0.1.0-py3-none-any.whl'), ktxDaemonSdist: join(root, 'ktx_daemon-0.1.0.tar.gz'), }); ``` - [ ] **Step 7: Update manifest file count expectations** In the `verifyArtifactManifest` test, replace: ```javascript assert.equal(manifest.files.length, NPM_ARTIFACT_PACKAGES.length + 4); ``` with: ```javascript assert.equal(manifest.files.length, NPM_ARTIFACT_PACKAGES.length + 5); ``` - [ ] **Step 8: Add CLI asset copy test** Add this test near the other artifact helper tests: ```javascript describe('copyRuntimeWheelAssets', () => { it('copies the runtime wheel and checksum manifest into CLI assets', async () => { const root = await mkdtemp(join(tmpdir(), 'ktx-runtime-assets-test-')); const layout = packageArtifactLayout(root); try { await mkdir(layout.pythonDir, { recursive: true }); await writeFile( join(layout.pythonDir, 'kaelio_ktx-0.1.0-py3-none-any.whl'), 'kaelio-ktx-runtime-wheel', ); const assets = await copyRuntimeWheelAssets(layout, { runtimeWheel: join(layout.pythonDir, 'kaelio_ktx-0.1.0-py3-none-any.whl'), }); assert.equal( assets.wheelPath, join(root, 'packages', 'cli', 'assets', 'python', 'kaelio_ktx-0.1.0-py3-none-any.whl'), ); assert.equal( assets.manifestPath, join(root, 'packages', 'cli', 'assets', 'python', CLI_PYTHON_ASSET_MANIFEST), ); const manifest = JSON.parse(await readFile(assets.manifestPath, 'utf8')); assert.deepEqual(manifest, { schemaVersion: 1, distributionName: RUNTIME_WHEEL_DISTRIBUTION_NAME, normalizedName: RUNTIME_WHEEL_NORMALIZED_NAME, version: RUNTIME_WHEEL_PACKAGE_VERSION, wheel: { file: 'kaelio_ktx-0.1.0-py3-none-any.whl', sha256: createHash('sha256') .update('kaelio-ktx-runtime-wheel') .digest('hex'), bytes: Buffer.byteLength('kaelio-ktx-runtime-wheel'), }, }); } finally { await rm(root, { recursive: true, force: true }); } }); }); ``` - [ ] **Step 9: Update install argument test** Replace the `pythonArtifactInstallArgs` expectation with one runtime wheel: ```javascript assert.deepEqual(args, [ 'pip', 'install', '--python', '/tmp/smoke/.venv/bin/python', '/repo/ktx/dist/artifacts/python/kaelio_ktx-0.1.0-py3-none-any.whl', ]); assert.equal(args.includes('ktx-daemon'), false); assert.equal(args.includes('ktx-sl'), false); assert.equal(args.includes('--find-links'), false); ``` - [ ] **Step 10: Run the failing package artifact tests** Run: ```bash node --test scripts/package-artifacts.test.mjs ``` Expected: FAIL with missing exports from `scripts/package-artifacts.mjs`. ### Task 5: Wire the runtime wheel into artifact packaging **Files:** - Modify: `scripts/package-artifacts.mjs` - Modify: `scripts/package-artifacts.test.mjs` - Test: `scripts/package-artifacts.test.mjs` - [ ] **Step 1: Import runtime wheel builder constants** Add this import near the top of `scripts/package-artifacts.mjs`: ```javascript import { RUNTIME_WHEEL_DISTRIBUTION_NAME, RUNTIME_WHEEL_NORMALIZED_NAME, RUNTIME_WHEEL_PACKAGE_VERSION, } from './build-python-runtime-wheel.mjs'; ``` Then re-export those constants after the existing constants: ```javascript export { RUNTIME_WHEEL_DISTRIBUTION_NAME, RUNTIME_WHEEL_NORMALIZED_NAME, RUNTIME_WHEEL_PACKAGE_VERSION, }; ``` - [ ] **Step 2: Add CLI asset manifest constant** Add this constant after `PYTHON_PACKAGE_VERSION`: ```javascript export const CLI_PYTHON_ASSET_MANIFEST = 'manifest.json'; ``` - [ ] **Step 3: Change build command order** Replace `buildArtifactCommands(layout)` with this implementation: ```javascript export function buildArtifactCommands(layout) { const npmBuildCommands = NPM_ARTIFACT_PACKAGES.map((packageInfo) => ({ command: 'pnpm', args: ['--filter', packageInfo.name, 'run', 'build'], cwd: layout.rootDir, })); const npmPackCommands = NPM_ARTIFACT_PACKAGES.map((packageInfo) => ({ command: 'pnpm', args: ['--filter', packageInfo.name, 'pack', '--out', layout.npmTarballs[packageInfo.name]], cwd: layout.rootDir, })); return [ ...npmBuildCommands, { command: process.execPath, args: ['scripts/build-python-runtime-wheel.mjs'], cwd: layout.rootDir, }, { command: 'uv', args: ['build', '--package', 'ktx-sl', '--out-dir', layout.pythonDir], cwd: layout.rootDir, }, { command: 'uv', args: ['build', '--package', 'ktx-daemon', '--out-dir', layout.pythonDir], cwd: layout.rootDir, }, ...npmPackCommands, ]; } ``` - [ ] **Step 4: Discover the runtime wheel** Update `findPythonArtifacts(pythonDir)` to return `runtimeWheel`: ```javascript export async function findPythonArtifacts(pythonDir) { const files = await readdir(pythonDir); return { runtimeWheel: findOne( files, RUNTIME_WHEEL_DISTRIBUTION_NAME, '.whl', 'kaelio-ktx runtime wheel', pythonDir, RUNTIME_WHEEL_PACKAGE_VERSION, ), ktxSlWheel: findOne(files, 'ktx-sl', '.whl', 'ktx-sl wheel', pythonDir), ktxSlSdist: findOne(files, 'ktx-sl', '.tar.gz', 'ktx-sl source distribution', pythonDir), ktxDaemonWheel: findOne(files, 'ktx-daemon', '.whl', 'ktx-daemon wheel', pythonDir), ktxDaemonSdist: findOne(files, 'ktx-daemon', '.tar.gz', 'ktx-daemon source distribution', pythonDir), }; } ``` Change `findOne` to accept an optional version: ```javascript function findOne(files, distributionName, suffix, label, pythonDir, version = PYTHON_PACKAGE_VERSION) { const normalized = normalizePythonDistributionName(distributionName); const found = files.find((file) => file.startsWith(`${normalized}-${version}`) && file.endsWith(suffix)); if (!found) { throw new Error(`Missing Python artifact: ${label}`); } return join(pythonDir, found); } ``` - [ ] **Step 5: Add runtime wheel release metadata** In `packageReleaseMetadata`, append this entry after `ktxDaemonPackage`: ```javascript releaseMetadataEntry({ ecosystem: 'python', packageName: RUNTIME_WHEEL_DISTRIBUTION_NAME, packageRoot: 'python/runtime-wheel', packageVersion: RUNTIME_WHEEL_PACKAGE_VERSION, privatePackage: false, }), ``` - [ ] **Step 6: Add runtime wheel to artifact manifest records** In `artifactPackageRecords`, add this record after npm records: ```javascript { artifactKind: 'wheel', artifactPath: pythonArtifacts.runtimeWheel, metadata: requirePackageMetadata(packagesByName, RUNTIME_WHEEL_DISTRIBUTION_NAME), }, ``` - [ ] **Step 7: Add CLI Python asset copy helper** Add this function before `pythonArtifactInstallArgs`: ```javascript function runtimeWheelAssetName(runtimeWheelPath) { return runtimeWheelPath.split(sep).at(-1); } export async function copyRuntimeWheelAssets(layout, pythonArtifacts) { const assetDir = join(layout.rootDir, 'packages', 'cli', 'assets', 'python'); const wheelFile = runtimeWheelAssetName(pythonArtifacts.runtimeWheel); if (!wheelFile) { throw new Error(`Unable to determine runtime wheel filename: ${pythonArtifacts.runtimeWheel}`); } const wheelContents = await readFile(pythonArtifacts.runtimeWheel); await rm(assetDir, { recursive: true, force: true }); await mkdir(assetDir, { recursive: true }); const wheelPath = join(assetDir, wheelFile); const manifestPath = join(assetDir, CLI_PYTHON_ASSET_MANIFEST); await writeFile(wheelPath, wheelContents); await writeFile( manifestPath, `${JSON.stringify( { schemaVersion: 1, distributionName: RUNTIME_WHEEL_DISTRIBUTION_NAME, normalizedName: RUNTIME_WHEEL_NORMALIZED_NAME, version: RUNTIME_WHEEL_PACKAGE_VERSION, wheel: { file: wheelFile, sha256: createHash('sha256').update(wheelContents).digest('hex'), bytes: wheelContents.byteLength, }, }, null, 2, )}\n`, ); return { assetDir, wheelPath, manifestPath }; } ``` - [ ] **Step 8: Install the runtime wheel in artifact smokes** Replace `pythonArtifactInstallArgs` with: ```javascript export function pythonArtifactInstallArgs(python, pythonArtifacts) { return ['pip', 'install', '--python', python, pythonArtifacts.runtimeWheel]; } ``` Update `pythonVerifySource()` to assert `kaelio-ktx` metadata and keep module imports: ```javascript export function pythonVerifySource() { return ` import importlib.metadata import semantic_layer import ktx_daemon assert importlib.metadata.version("kaelio-ktx") == "0.1.0" assert semantic_layer is not None assert ktx_daemon.PACKAGE_NAME == "ktx-daemon" `; } ``` - [ ] **Step 9: Copy runtime assets before npm packing** Replace the loop in `buildArtifacts(layout)` with these explicit phases: ```javascript const commands = buildArtifactCommands(layout); const npmBuildCount = NPM_ARTIFACT_PACKAGES.length; const npmPackStart = commands.length - NPM_ARTIFACT_PACKAGES.length; for (const command of commands.slice(0, npmBuildCount)) { await runCommand(command.command, command.args, { cwd: command.cwd }); } for (const command of commands.slice(npmBuildCount, npmPackStart)) { await runCommand(command.command, command.args, { cwd: command.cwd }); } const pythonArtifacts = await findPythonArtifacts(layout.pythonDir); await copyRuntimeWheelAssets(layout, pythonArtifacts); for (const command of commands.slice(npmPackStart)) { await runCommand(command.command, command.args, { cwd: command.cwd }); } ``` - [ ] **Step 10: Run package artifact tests** Run: ```bash node --test scripts/package-artifacts.test.mjs ``` Expected: PASS. - [ ] **Step 11: Commit artifact wiring** Run: ```bash git add scripts/package-artifacts.mjs scripts/package-artifacts.test.mjs git commit -m "build: bundle python runtime wheel in cli artifacts" ``` ### Task 6: Update release policy and generated asset ignores **Files:** - Modify: `release-policy.json` - Modify: `.gitignore` - Modify: `scripts/release-readiness.test.mjs` - Test: `scripts/release-readiness.test.mjs` - [ ] **Step 1: Ignore generated CLI Python assets** Add this block to `.gitignore` after the `dist/` ignore: ```gitignore packages/cli/assets/python/ ``` - [ ] **Step 2: Add runtime wheel to release policy** Update `release-policy.json` so the Python packages list is: ```json "python": { "publish": false, "repository": null, "packages": ["ktx-sl", "ktx-daemon", "kaelio-ktx"] }, ``` - [ ] **Step 3: Update release readiness fixtures** In `scripts/release-readiness.test.mjs`, update fixture policy objects that list Python packages from: ```javascript packages: ['ktx-sl', 'ktx-daemon'], ``` to: ```javascript packages: ['ktx-sl', 'ktx-daemon', 'kaelio-ktx'], ``` Update expected package name arrays to include `kaelio-ktx`: ```javascript packageNames: [ ...NPM_ARTIFACT_PACKAGES.map((packageInfo) => packageInfo.name), 'ktx-sl', 'ktx-daemon', 'kaelio-ktx', ], ``` - [ ] **Step 4: Run release readiness tests** Run: ```bash node --test scripts/release-readiness.test.mjs ``` Expected: PASS. - [ ] **Step 5: Commit policy updates** Run: ```bash git add .gitignore release-policy.json scripts/release-readiness.test.mjs git commit -m "build: track bundled python runtime release artifact" ``` ### Task 7: Verify the built runtime wheel end to end **Files:** - Build output: `dist/artifacts/python/kaelio_ktx-0.1.0-py3-none-any.whl` - Build output: `packages/cli/assets/python/manifest.json` - Build output: `packages/cli/assets/python/kaelio_ktx-0.1.0-py3-none-any.whl` - [ ] **Step 1: Run focused script tests** Run: ```bash node --test scripts/build-python-runtime-wheel.test.mjs scripts/package-artifacts.test.mjs scripts/release-readiness.test.mjs ``` Expected: PASS. - [ ] **Step 2: Run Python package tests affected by dependency split** Run: ```bash uv run pytest python/ktx-daemon/tests -q ``` Expected: PASS. - [ ] **Step 3: Run package artifact check** Run: ```bash pnpm run artifacts:check ``` Expected: PASS. This command builds the runtime wheel, copies it into CLI assets before npm packing, installs the packed npm packages in a clean smoke project, installs the bundled runtime wheel with `uv pip install`, and verifies `semantic_layer` plus `ktx_daemon` imports from the one `kaelio-ktx` wheel. - [ ] **Step 4: Inspect the generated CLI asset manifest** Run: ```bash node -e "const fs=require('node:fs'); const m=JSON.parse(fs.readFileSync('packages/cli/assets/python/manifest.json','utf8')); console.log(m.distributionName, m.version, m.wheel.file, m.wheel.sha256.length)" ``` Expected output: ```text kaelio-ktx 0.1.0 kaelio_ktx-0.1.0-py3-none-any.whl 64 ``` - [ ] **Step 5: Run pre-commit when configured** Run this only if `.pre-commit-config.yaml` exists: ```bash uv run pre-commit run --files python/ktx-daemon/pyproject.toml uv.lock pyproject.toml scripts/build-python-runtime-wheel.mjs scripts/build-python-runtime-wheel.test.mjs scripts/package-artifacts.mjs scripts/package-artifacts.test.mjs scripts/release-readiness.test.mjs release-policy.json .gitignore ``` Expected: PASS. If no pre-commit config exists, record that no pre-commit configuration exists in this repository and skip this command. - [ ] **Step 6: Commit verification-only updates if any** If verification required small code or test fixes, commit them: ```bash git add scripts/build-python-runtime-wheel.mjs scripts/build-python-runtime-wheel.test.mjs scripts/package-artifacts.mjs scripts/package-artifacts.test.mjs scripts/release-readiness.test.mjs python/ktx-daemon/pyproject.toml uv.lock release-policy.json .gitignore git commit -m "test: verify bundled python runtime wheel" ``` If no files changed after verification, do not create an empty commit. ## Acceptance criteria - `dist/artifacts/python/kaelio_ktx-0.1.0-py3-none-any.whl` is built by `pnpm run artifacts:check`. - The built CLI npm tarball includes `assets/python/kaelio_ktx-0.1.0-py3-none-any.whl` and `assets/python/manifest.json`. - The asset manifest records the wheel filename, byte count, and SHA-256. - Installing only the bundled runtime wheel exposes `semantic_layer`, `ktx_daemon`, and the `ktx-daemon` console script. - `sentence-transformers` and `torch` are absent from default dependencies and present under the `local-embeddings` extra. - Existing separate `ktx-sl` and `ktx-daemon` artifacts can remain CI artifacts in this plan; the npm runtime payload uses `kaelio-ktx`. ## Self-review Spec coverage: - Covers the package-model requirement for one bundled KTX-owned Python wheel. - Covers the wheel checksum or runtime manifest requirement by adding the npm asset manifest. - Covers lazy local embedding dependencies by moving heavy packages into the `local-embeddings` extra. - Leaves managed runtime directories, install commands, daemon reuse, and `@kaelio/ktx` npm renaming for later plans. Placeholder scan: - The plan contains no placeholder markers and no unspecified implementation steps. Type and name consistency: - Runtime distribution name is consistently `kaelio-ktx`. - Wheel filename prefix is consistently `kaelio_ktx`. - Runtime version is consistently `0.1.0`.