* docs: add npm managed python runtime design * build: add bundled python runtime wheel builder * build: make local embedding dependencies optional * build: bundle python runtime wheel in cli artifacts * build: track bundled python runtime release artifact * test: verify bundled python runtime wheel * docs: add plan for bundled python runtime wheel * test: cover managed python runtime lifecycle * feat: add managed python runtime installer * feat: add runtime command runner * feat: expose runtime management commands * test: verify managed python runtime commands * docs: add plan for managed python runtime installer * feat: add managed python command helper * feat: use managed runtime for sl query compute * feat: route sl query managed runtime policy * docs: add plan for managed runtime sl query integration * feat: add managed runtime daemon metadata * feat: manage python daemon lifecycle * feat: add runtime daemon start stop commands * fix: verify managed runtime daemon lifecycle * docs: add plan for managed runtime daemon lifecycle * feat: add managed local embeddings config marker * feat: add managed local embeddings daemon helper * feat: use managed runtime for local embedding setup * feat: pass managed runtime policy through setup * docs: add plan for managed local embeddings runtime * feat: read CLI package metadata dynamically * feat: assemble public kaelio ktx npm package * feat: release one public kaelio ktx npm artifact * test: cover public kaelio ktx package invocations * chore: verify public kaelio ktx package artifacts * docs: add plan for public kaelio ktx npm package * test: verify managed runtime in public package smoke * test: finalize managed runtime release smoke * docs: add plan for managed runtime release smoke * test: specify local embeddings release smoke * feat: add local embeddings runtime smoke * chore: register local embeddings smoke * fix: verify local embeddings smoke * fix: restore artifact smoke python env helper * docs: add plan for managed local embeddings release smoke * refactor: share managed runtime install policy parsing * feat: use managed runtime for agent semantic queries * feat: use managed runtime for MCP semantic compute * docs: add plan for managed agent and MCP semantic runtime * feat(cli): add managed daemon HTTP helpers * feat(cli): route local adapters through managed daemon * feat(cli): use managed daemon for ingest helpers * feat(cli): pass managed daemon options to scan * feat(context): pass MCP ingest pull config options * feat(cli): pass managed daemon options to serve ingest * test: verify managed local ingest daemon runtime * docs: add plan for managed local ingest daemon runtime * docs: align managed runtime examples * docs: add plan for managed runtime docs cleanup * test: cover published package runtime smoke commands * test: validate published package smoke outputs * docs: add plan for published package runtime smoke * build: stamp public npm package version * release: add npm public release policy * release: add guarded npm publish script * release: document public npm release handoff * docs: add plan for public npm release handoff * test: cover managed runtime prune in package smoke * docs: document managed runtime prune * docs: add plan for managed runtime prune smoke and docs * chore: encode uv runtime prerequisite policy * fix: clarify missing uv runtime error * docs: document uv runtime prerequisite * docs: add plan for uv runtime prerequisite contract * refactor: limit release artifacts to public package runtime * chore: align release policy with bundled runtime wheel * docs: describe single public runtime artifact surface * test: verify single public runtime artifact contract * docs: add plan for single public runtime artifact cleanup * fix: align local embeddings smoke with public version * docs: add plan for local embeddings smoke public version * release: soft-launch as @kaelio/ktx@0.1.0-rc.0 on next tag Publish target moves to the pre-release version 0.1.0-rc.0 under the next dist-tag so npm install @kaelio/ktx (which resolves to latest) does not pick up the soft-launch build. Users opt in via @kaelio/ktx@next. * Fix release script boundary checks * Remove PostHog from public package bundle
33 KiB
Bundled Python Runtime Wheel Implementation Plan
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Build and package one bundled kaelio-ktx Python wheel that contains
KTX-owned Python runtime code and keeps local embedding dependencies optional.
Architecture: Add a deterministic Node assembly script that copies the
existing semantic_layer and ktx_daemon source trees into a temporary wheel
source tree, writes a runtime-only pyproject.toml, and builds one wheel with
uv build. Wire package artifacts so the CLI npm tarball includes the bundled
wheel plus a checksum manifest under assets/python/.
Tech Stack: Node 22 ESM scripts, node:test, uv, Hatchling, Python 3.13,
pnpm, TypeScript package artifacts.
Existing status
This plan is based on
docs/superpowers/specs/2026-05-11-npm-managed-python-runtime-design.md.
There are no committed plan files under docs/superpowers/plans/ in this
worktree or in git history for this spec. The spec itself is the only tracked
Superpowers document.
The following pieces are already implemented:
packages/context/src/daemon/semantic-layer-compute.tscan invokepython -m ktx_daemonfor one-shot semantic-layer operations.python/ktx-daemonexposesktx-daemonone-shot commands and an HTTPserve-httpdaemon with/health.scripts/package-artifacts.mjsbuilds npm package tarballs and separatektx-slandktx-daemonPython artifacts.scripts/package-artifacts.mjswrites a checksummed artifact manifest.
The following spec requirements are not implemented yet:
- A single public
@kaelio/ktxnpm surface. - One KTX-owned bundled Python wheel inside the npm package.
- A managed runtime root, installer, runtime manifest, and runtime command family.
- Lazy
local-embeddingsinstallation that keepssentence-transformersandtorchout of the default Python dependency set.
This plan implements the bundled wheel prerequisite. Runtime install commands must be planned after this lands because they need a real wheel payload and checksum manifest to install.
File structure
- Create
scripts/build-python-runtime-wheel.mjs: assembles the temporary runtime wheel source tree and runsuv build. - Create
scripts/build-python-runtime-wheel.test.mjs: tests source copying, generatedpyproject.toml, and theuv buildcommand shape. - Modify
scripts/package-artifacts.mjs: builds the runtime wheel before npm packing, copies it intopackages/cli/assets/python/, includes it in the artifact manifest, and installs it in artifact smoke tests. - Modify
scripts/package-artifacts.test.mjs: covers runtime wheel metadata, manifest entries, install arguments, and CLI asset copy behavior. - Modify
scripts/release-readiness.test.mjs: expectskaelio-ktxin Python release metadata and policy fixtures. - Modify
release-policy.json: listskaelio-ktxas a CI-only Python artifact. - Modify
python/ktx-daemon/pyproject.toml: movessentence-transformersandtorchto alocal-embeddingsoptional dependency group. - Modify
uv.lock: records the dependency metadata change. - Modify
.gitignore: ignores generatedpackages/cli/assets/python/contents.
Plan status
No earlier plans were found for this spec. This is plan 1 for the spec.
Task 1: Add failing tests for the runtime wheel builder
Files:
-
Create:
scripts/build-python-runtime-wheel.test.mjs -
Test:
scripts/build-python-runtime-wheel.test.mjs -
Step 1: Write the failing test file
Create scripts/build-python-runtime-wheel.test.mjs with this content:
import assert from 'node:assert/strict';
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { describe, it } from 'node:test';
import {
RUNTIME_WHEEL_DISTRIBUTION_NAME,
RUNTIME_WHEEL_PACKAGE_VERSION,
createRuntimeWheelBuildTree,
runtimeWheelBuildCommand,
runtimeWheelLayout,
runtimeWheelPyproject,
} from './build-python-runtime-wheel.mjs';
async function writeRuntimeSourceFixture(root) {
await mkdir(join(root, 'python', 'ktx-sl', 'semantic_layer'), {
recursive: true,
});
await mkdir(join(root, 'python', 'ktx-daemon', 'src', 'ktx_daemon'), {
recursive: true,
});
await writeFile(
join(root, 'python', 'ktx-sl', 'semantic_layer', '__init__.py'),
'SEMANTIC_LAYER_FIXTURE = True\n',
);
await writeFile(
join(root, 'python', 'ktx-daemon', 'src', 'ktx_daemon', '__init__.py'),
'KTX_DAEMON_FIXTURE = True\n',
);
await writeFile(
join(root, 'python', 'ktx-daemon', 'src', 'ktx_daemon', '__main__.py'),
'def main():\n return 0\n',
);
}
describe('runtimeWheelLayout', () => {
it('uses stable source, build, and output paths', () => {
const layout = runtimeWheelLayout('/repo/ktx');
assert.equal(layout.rootDir, '/repo/ktx');
assert.equal(layout.semanticLayerSourceDir, '/repo/ktx/python/ktx-sl/semantic_layer');
assert.equal(layout.daemonSourceDir, '/repo/ktx/python/ktx-daemon/src/ktx_daemon');
assert.equal(layout.buildRoot, '/repo/ktx/dist/runtime-wheel-src');
assert.equal(layout.outputDir, '/repo/ktx/dist/artifacts/python');
});
});
describe('runtimeWheelPyproject', () => {
it('describes one kaelio-ktx wheel with lazy local embeddings', () => {
const pyproject = runtimeWheelPyproject();
assert.match(pyproject, /name = "kaelio-ktx"/);
assert.match(pyproject, /version = "0\.1\.0"/);
assert.match(pyproject, /ktx-daemon = "ktx_daemon\.__main__:main"/);
assert.match(pyproject, /packages = \["semantic_layer", "ktx_daemon"\]/);
assert.match(pyproject, /\[project\.optional-dependencies\]/);
assert.match(pyproject, /local-embeddings = \[/);
assert.match(pyproject, /"sentence-transformers>=5\.1\.1"/);
assert.match(pyproject, /"torch>=2\.2\.0"/);
assert.doesNotMatch(
pyproject.match(/dependencies = \[[\s\S]*?\]/)?.[0] ?? '',
/sentence-transformers|torch/,
);
});
});
describe('createRuntimeWheelBuildTree', () => {
it('copies KTX-owned Python packages into the build tree', async () => {
const root = await mkdtemp(join(tmpdir(), 'ktx-runtime-wheel-test-'));
try {
await writeRuntimeSourceFixture(root);
const layout = runtimeWheelLayout(root);
await createRuntimeWheelBuildTree(layout);
assert.equal(
await readFile(join(layout.buildRoot, 'semantic_layer', '__init__.py'), 'utf8'),
'SEMANTIC_LAYER_FIXTURE = True\n',
);
assert.equal(
await readFile(join(layout.buildRoot, 'ktx_daemon', '__main__.py'), 'utf8'),
'def main():\n return 0\n',
);
const pyproject = await readFile(join(layout.buildRoot, 'pyproject.toml'), 'utf8');
assert.match(pyproject, /name = "kaelio-ktx"/);
assert.match(pyproject, /local-embeddings = \[/);
const readme = await readFile(join(layout.buildRoot, 'README.md'), 'utf8');
assert.match(readme, /Bundled Python runtime wheel for KTX/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
});
describe('runtimeWheelBuildCommand', () => {
it('runs uv build against the generated build tree', () => {
const layout = runtimeWheelLayout('/repo/ktx');
assert.deepEqual(runtimeWheelBuildCommand(layout), {
command: 'uv',
args: [
'build',
'--wheel',
'--out-dir',
'/repo/ktx/dist/artifacts/python',
'/repo/ktx/dist/runtime-wheel-src',
],
cwd: '/repo/ktx',
});
assert.equal(RUNTIME_WHEEL_DISTRIBUTION_NAME, 'kaelio-ktx');
assert.equal(RUNTIME_WHEEL_PACKAGE_VERSION, '0.1.0');
});
});
- Step 2: Run the failing test
Run:
node --test scripts/build-python-runtime-wheel.test.mjs
Expected: FAIL with an import error for
./build-python-runtime-wheel.mjs.
Task 2: Implement the runtime wheel builder
Files:
-
Create:
scripts/build-python-runtime-wheel.mjs -
Test:
scripts/build-python-runtime-wheel.test.mjs -
Step 1: Create the builder script
Create scripts/build-python-runtime-wheel.mjs with this content:
#!/usr/bin/env node
import { execFile } from 'node:child_process';
import { cp, mkdir, readFile, rm, writeFile } from 'node:fs/promises';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { promisify } from 'node:util';
const execFileAsync = promisify(execFile);
export const RUNTIME_WHEEL_DISTRIBUTION_NAME = 'kaelio-ktx';
export const RUNTIME_WHEEL_NORMALIZED_NAME = 'kaelio_ktx';
export const RUNTIME_WHEEL_PACKAGE_VERSION = '0.1.0';
function scriptRootDir() {
return resolve(dirname(fileURLToPath(import.meta.url)), '..');
}
export function runtimeWheelLayout(rootDir = scriptRootDir()) {
return {
rootDir,
semanticLayerSourceDir: join(rootDir, 'python', 'ktx-sl', 'semantic_layer'),
daemonSourceDir: join(rootDir, 'python', 'ktx-daemon', 'src', 'ktx_daemon'),
buildRoot: join(rootDir, 'dist', 'runtime-wheel-src'),
outputDir: join(rootDir, 'dist', 'artifacts', 'python'),
};
}
export function runtimeWheelPyproject() {
return `[project]
name = "${RUNTIME_WHEEL_DISTRIBUTION_NAME}"
version = "${RUNTIME_WHEEL_PACKAGE_VERSION}"
description = "Bundled Python runtime payload for the KTX npm package"
readme = "README.md"
requires-python = ">=3.13"
license = "Apache-2.0"
dependencies = [
"fastapi>=0.115.0",
"lkml>=1.3.7",
"numpy>=2.2.6",
"orjson>=3.11.4",
"pandas>=2.2.3",
"psycopg[binary]>=3.2.0",
"pydantic>=2.9.0",
"pyyaml>=6",
"requests>=2.32.0",
"sqlglot>=26",
"uvicorn[standard]>=0.32.0",
]
[project.optional-dependencies]
local-embeddings = [
"sentence-transformers>=5.1.1",
"torch>=2.2.0",
]
[project.scripts]
ktx-daemon = "ktx_daemon.__main__:main"
[project.urls]
Homepage = "https://github.com/kaelio/ktx"
Repository = "https://github.com/kaelio/ktx"
Issues = "https://github.com/kaelio/ktx/issues"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["semantic_layer", "ktx_daemon"]
`;
}
export function runtimeWheelReadme() {
return `# kaelio-ktx Python runtime
Bundled Python runtime wheel for KTX.
This wheel is built from the repository's \`semantic_layer\` and
\`ktx_daemon\` source trees for inclusion in the npm package. It is not a
separate public PyPI release artifact.
`;
}
export async function createRuntimeWheelBuildTree(layout = runtimeWheelLayout()) {
await rm(layout.buildRoot, { recursive: true, force: true });
await mkdir(layout.buildRoot, { recursive: true });
await cp(layout.semanticLayerSourceDir, join(layout.buildRoot, 'semantic_layer'), {
recursive: true,
});
await cp(layout.daemonSourceDir, join(layout.buildRoot, 'ktx_daemon'), {
recursive: true,
});
await writeFile(join(layout.buildRoot, 'pyproject.toml'), runtimeWheelPyproject());
await writeFile(join(layout.buildRoot, 'README.md'), runtimeWheelReadme());
}
export function runtimeWheelBuildCommand(layout = runtimeWheelLayout()) {
return {
command: 'uv',
args: ['build', '--wheel', '--out-dir', layout.outputDir, layout.buildRoot],
cwd: layout.rootDir,
};
}
async function runCommand(command, args, options) {
const result = await execFileAsync(command, args, {
cwd: options.cwd,
encoding: 'utf8',
maxBuffer: 1024 * 1024 * 20,
});
if (result.stdout) {
process.stdout.write(result.stdout);
}
if (result.stderr) {
process.stderr.write(result.stderr);
}
}
export async function buildRuntimeWheel(layout = runtimeWheelLayout()) {
await mkdir(layout.outputDir, { recursive: true });
await createRuntimeWheelBuildTree(layout);
const command = runtimeWheelBuildCommand(layout);
await runCommand(command.command, command.args, { cwd: command.cwd });
const pyproject = await readFile(join(layout.buildRoot, 'pyproject.toml'), 'utf8');
return {
buildRoot: layout.buildRoot,
outputDir: layout.outputDir,
pyproject,
};
}
async function main() {
await buildRuntimeWheel(runtimeWheelLayout());
}
if (import.meta.url === pathToFileURL(process.argv[1] ?? '').href) {
try {
await main();
} catch (error) {
process.stderr.write(`${error instanceof Error ? error.stack : String(error)}\n`);
process.exitCode = 1;
}
}
- Step 2: Run the builder test
Run:
node --test scripts/build-python-runtime-wheel.test.mjs
Expected: PASS.
- Step 3: Commit the builder
Run:
git add scripts/build-python-runtime-wheel.mjs scripts/build-python-runtime-wheel.test.mjs
git commit -m "build: add bundled python runtime wheel builder"
Task 3: Move heavy local embedding dependencies behind an extra
Files:
-
Modify:
python/ktx-daemon/pyproject.toml -
Modify:
uv.lock -
Test:
python/ktx-daemon/tests/test_embeddings.py -
Test:
scripts/build-python-runtime-wheel.test.mjs -
Step 1: Update daemon dependencies
In python/ktx-daemon/pyproject.toml, remove these two lines from
[project].dependencies:
"sentence-transformers>=5.1.1",
"torch>=2.2.0",
Add this block immediately after [project.scripts]:
[project.optional-dependencies]
local-embeddings = [
"sentence-transformers>=5.1.1",
"torch>=2.2.0",
]
The relevant section must read:
[project]
name = "ktx-daemon"
version = "0.1.0"
description = "Portable compute package for KTX semantic-layer operations"
readme = "README.md"
requires-python = ">=3.13"
license = "Apache-2.0"
dependencies = [
"fastapi>=0.115.0",
"ktx-sl",
"lkml>=1.3.7",
"numpy>=2.2.6",
"orjson>=3.11.4",
"pandas>=2.2.3",
"psycopg[binary]>=3.2.0",
"pydantic>=2.9.0",
"requests>=2.32.0",
"sqlglot>=26",
"uvicorn[standard]>=0.32.0",
]
[project.scripts]
ktx-daemon = "ktx_daemon.__main__:main"
[project.optional-dependencies]
local-embeddings = [
"sentence-transformers>=5.1.1",
"torch>=2.2.0",
]
- Step 2: Refresh the uv lockfile
Run:
uv lock
Expected: PASS and uv.lock records the ktx-daemon optional dependency
metadata. If the local uv version is older than tool.uv.required-version,
record the version mismatch and do not edit pyproject.toml to lower the pin.
- Step 3: Run Python tests that cover lazy embedding imports
Run:
uv run pytest python/ktx-daemon/tests/test_embeddings.py -q
Expected: PASS. The tests use injected fake providers and do not require
sentence-transformers or torch.
- Step 4: Run the runtime wheel metadata test
Run:
node --test scripts/build-python-runtime-wheel.test.mjs
Expected: PASS and the generated runtime pyproject.toml keeps
sentence-transformers and torch under local-embeddings.
- Step 5: Commit the dependency split
Run:
git add python/ktx-daemon/pyproject.toml uv.lock
git commit -m "build: make local embedding dependencies optional"
Task 4: Add artifact tests for the bundled runtime wheel
Files:
-
Modify:
scripts/package-artifacts.test.mjs -
Test:
scripts/package-artifacts.test.mjs -
Step 1: Extend imports
In scripts/package-artifacts.test.mjs, extend the import from
./package-artifacts.mjs with these names:
CLI_PYTHON_ASSET_MANIFEST,
RUNTIME_WHEEL_DISTRIBUTION_NAME,
RUNTIME_WHEEL_NORMALIZED_NAME,
RUNTIME_WHEEL_PACKAGE_VERSION,
copyRuntimeWheelAssets,
- Step 2: Update Python metadata fixtures
In writeReleaseMetadataInputs, keep the existing ktx-sl and ktx-daemon
fixture files and add no new on-disk Python package. The runtime wheel metadata
will come from constants exported by package-artifacts.mjs.
- Step 3: Update uploadable artifact fixtures
In writeUploadableArtifactFixtures, add this runtime wheel entry to
fileContents:
[
join(layout.pythonDir, 'kaelio_ktx-0.1.0-py3-none-any.whl'),
'kaelio-ktx-runtime-wheel',
],
- Step 4: Update build command expectations
Replace the buildArtifactCommands expectations with these three assertions:
assert.deepEqual(
commands.slice(0, NPM_ARTIFACT_PACKAGES.length).map((command) => [command.command, command.args]),
NPM_ARTIFACT_PACKAGES.map((packageInfo) => ['pnpm', ['--filter', packageInfo.name, 'run', 'build']]),
);
assert.deepEqual(
commands
.slice(NPM_ARTIFACT_PACKAGES.length, NPM_ARTIFACT_PACKAGES.length + 3)
.map((command) => [command.command, command.args]),
[
[
process.execPath,
['scripts/build-python-runtime-wheel.mjs'],
],
[
'uv',
['build', '--package', 'ktx-sl', '--out-dir', '/repo/ktx/dist/artifacts/python'],
],
[
'uv',
['build', '--package', 'ktx-daemon', '--out-dir', '/repo/ktx/dist/artifacts/python'],
],
],
);
assert.deepEqual(
commands.slice(NPM_ARTIFACT_PACKAGES.length + 3).map((command) => [command.command, command.args]),
NPM_ARTIFACT_PACKAGES.map((packageInfo) => [
'pnpm',
['--filter', packageInfo.name, 'pack', '--out', layout.npmTarballs[packageInfo.name]],
]),
);
- Step 5: Update release metadata expectations
In the packageReleaseMetadata test, add this Python metadata entry after
ktx-daemon:
{
ecosystem: 'python',
packageName: 'kaelio-ktx',
packageRoot: 'python/runtime-wheel',
packageVersion: '0.1.0',
private: false,
releaseMode: 'ci-artifact-only',
},
- Step 6: Update Python artifact discovery expectations
In the findPythonArtifacts test, create the runtime wheel fixture:
await writeFile(join(root, 'kaelio_ktx-0.1.0-py3-none-any.whl'), '');
Then update the expected object:
assert.deepEqual(await findPythonArtifacts(root), {
runtimeWheel: join(root, 'kaelio_ktx-0.1.0-py3-none-any.whl'),
ktxSlWheel: join(root, 'ktx_sl-0.1.0-py3-none-any.whl'),
ktxSlSdist: join(root, 'ktx_sl-0.1.0.tar.gz'),
ktxDaemonWheel: join(root, 'ktx_daemon-0.1.0-py3-none-any.whl'),
ktxDaemonSdist: join(root, 'ktx_daemon-0.1.0.tar.gz'),
});
- Step 7: Update manifest file count expectations
In the verifyArtifactManifest test, replace:
assert.equal(manifest.files.length, NPM_ARTIFACT_PACKAGES.length + 4);
with:
assert.equal(manifest.files.length, NPM_ARTIFACT_PACKAGES.length + 5);
- Step 8: Add CLI asset copy test
Add this test near the other artifact helper tests:
describe('copyRuntimeWheelAssets', () => {
it('copies the runtime wheel and checksum manifest into CLI assets', async () => {
const root = await mkdtemp(join(tmpdir(), 'ktx-runtime-assets-test-'));
const layout = packageArtifactLayout(root);
try {
await mkdir(layout.pythonDir, { recursive: true });
await writeFile(
join(layout.pythonDir, 'kaelio_ktx-0.1.0-py3-none-any.whl'),
'kaelio-ktx-runtime-wheel',
);
const assets = await copyRuntimeWheelAssets(layout, {
runtimeWheel: join(layout.pythonDir, 'kaelio_ktx-0.1.0-py3-none-any.whl'),
});
assert.equal(
assets.wheelPath,
join(root, 'packages', 'cli', 'assets', 'python', 'kaelio_ktx-0.1.0-py3-none-any.whl'),
);
assert.equal(
assets.manifestPath,
join(root, 'packages', 'cli', 'assets', 'python', CLI_PYTHON_ASSET_MANIFEST),
);
const manifest = JSON.parse(await readFile(assets.manifestPath, 'utf8'));
assert.deepEqual(manifest, {
schemaVersion: 1,
distributionName: RUNTIME_WHEEL_DISTRIBUTION_NAME,
normalizedName: RUNTIME_WHEEL_NORMALIZED_NAME,
version: RUNTIME_WHEEL_PACKAGE_VERSION,
wheel: {
file: 'kaelio_ktx-0.1.0-py3-none-any.whl',
sha256: createHash('sha256')
.update('kaelio-ktx-runtime-wheel')
.digest('hex'),
bytes: Buffer.byteLength('kaelio-ktx-runtime-wheel'),
},
});
} finally {
await rm(root, { recursive: true, force: true });
}
});
});
- Step 9: Update install argument test
Replace the pythonArtifactInstallArgs expectation with one runtime wheel:
assert.deepEqual(args, [
'pip',
'install',
'--python',
'/tmp/smoke/.venv/bin/python',
'/repo/ktx/dist/artifacts/python/kaelio_ktx-0.1.0-py3-none-any.whl',
]);
assert.equal(args.includes('ktx-daemon'), false);
assert.equal(args.includes('ktx-sl'), false);
assert.equal(args.includes('--find-links'), false);
- Step 10: Run the failing package artifact tests
Run:
node --test scripts/package-artifacts.test.mjs
Expected: FAIL with missing exports from scripts/package-artifacts.mjs.
Task 5: Wire the runtime wheel into artifact packaging
Files:
-
Modify:
scripts/package-artifacts.mjs -
Modify:
scripts/package-artifacts.test.mjs -
Test:
scripts/package-artifacts.test.mjs -
Step 1: Import runtime wheel builder constants
Add this import near the top of scripts/package-artifacts.mjs:
import {
RUNTIME_WHEEL_DISTRIBUTION_NAME,
RUNTIME_WHEEL_NORMALIZED_NAME,
RUNTIME_WHEEL_PACKAGE_VERSION,
} from './build-python-runtime-wheel.mjs';
Then re-export those constants after the existing constants:
export {
RUNTIME_WHEEL_DISTRIBUTION_NAME,
RUNTIME_WHEEL_NORMALIZED_NAME,
RUNTIME_WHEEL_PACKAGE_VERSION,
};
- Step 2: Add CLI asset manifest constant
Add this constant after PYTHON_PACKAGE_VERSION:
export const CLI_PYTHON_ASSET_MANIFEST = 'manifest.json';
- Step 3: Change build command order
Replace buildArtifactCommands(layout) with this implementation:
export function buildArtifactCommands(layout) {
const npmBuildCommands = NPM_ARTIFACT_PACKAGES.map((packageInfo) => ({
command: 'pnpm',
args: ['--filter', packageInfo.name, 'run', 'build'],
cwd: layout.rootDir,
}));
const npmPackCommands = NPM_ARTIFACT_PACKAGES.map((packageInfo) => ({
command: 'pnpm',
args: ['--filter', packageInfo.name, 'pack', '--out', layout.npmTarballs[packageInfo.name]],
cwd: layout.rootDir,
}));
return [
...npmBuildCommands,
{
command: process.execPath,
args: ['scripts/build-python-runtime-wheel.mjs'],
cwd: layout.rootDir,
},
{
command: 'uv',
args: ['build', '--package', 'ktx-sl', '--out-dir', layout.pythonDir],
cwd: layout.rootDir,
},
{
command: 'uv',
args: ['build', '--package', 'ktx-daemon', '--out-dir', layout.pythonDir],
cwd: layout.rootDir,
},
...npmPackCommands,
];
}
- Step 4: Discover the runtime wheel
Update findPythonArtifacts(pythonDir) to return runtimeWheel:
export async function findPythonArtifacts(pythonDir) {
const files = await readdir(pythonDir);
return {
runtimeWheel: findOne(
files,
RUNTIME_WHEEL_DISTRIBUTION_NAME,
'.whl',
'kaelio-ktx runtime wheel',
pythonDir,
RUNTIME_WHEEL_PACKAGE_VERSION,
),
ktxSlWheel: findOne(files, 'ktx-sl', '.whl', 'ktx-sl wheel', pythonDir),
ktxSlSdist: findOne(files, 'ktx-sl', '.tar.gz', 'ktx-sl source distribution', pythonDir),
ktxDaemonWheel: findOne(files, 'ktx-daemon', '.whl', 'ktx-daemon wheel', pythonDir),
ktxDaemonSdist: findOne(files, 'ktx-daemon', '.tar.gz', 'ktx-daemon source distribution', pythonDir),
};
}
Change findOne to accept an optional version:
function findOne(files, distributionName, suffix, label, pythonDir, version = PYTHON_PACKAGE_VERSION) {
const normalized = normalizePythonDistributionName(distributionName);
const found = files.find((file) => file.startsWith(`${normalized}-${version}`) && file.endsWith(suffix));
if (!found) {
throw new Error(`Missing Python artifact: ${label}`);
}
return join(pythonDir, found);
}
- Step 5: Add runtime wheel release metadata
In packageReleaseMetadata, append this entry after ktxDaemonPackage:
releaseMetadataEntry({
ecosystem: 'python',
packageName: RUNTIME_WHEEL_DISTRIBUTION_NAME,
packageRoot: 'python/runtime-wheel',
packageVersion: RUNTIME_WHEEL_PACKAGE_VERSION,
privatePackage: false,
}),
- Step 6: Add runtime wheel to artifact manifest records
In artifactPackageRecords, add this record after npm records:
{
artifactKind: 'wheel',
artifactPath: pythonArtifacts.runtimeWheel,
metadata: requirePackageMetadata(packagesByName, RUNTIME_WHEEL_DISTRIBUTION_NAME),
},
- Step 7: Add CLI Python asset copy helper
Add this function before pythonArtifactInstallArgs:
function runtimeWheelAssetName(runtimeWheelPath) {
return runtimeWheelPath.split(sep).at(-1);
}
export async function copyRuntimeWheelAssets(layout, pythonArtifacts) {
const assetDir = join(layout.rootDir, 'packages', 'cli', 'assets', 'python');
const wheelFile = runtimeWheelAssetName(pythonArtifacts.runtimeWheel);
if (!wheelFile) {
throw new Error(`Unable to determine runtime wheel filename: ${pythonArtifacts.runtimeWheel}`);
}
const wheelContents = await readFile(pythonArtifacts.runtimeWheel);
await rm(assetDir, { recursive: true, force: true });
await mkdir(assetDir, { recursive: true });
const wheelPath = join(assetDir, wheelFile);
const manifestPath = join(assetDir, CLI_PYTHON_ASSET_MANIFEST);
await writeFile(wheelPath, wheelContents);
await writeFile(
manifestPath,
`${JSON.stringify(
{
schemaVersion: 1,
distributionName: RUNTIME_WHEEL_DISTRIBUTION_NAME,
normalizedName: RUNTIME_WHEEL_NORMALIZED_NAME,
version: RUNTIME_WHEEL_PACKAGE_VERSION,
wheel: {
file: wheelFile,
sha256: createHash('sha256').update(wheelContents).digest('hex'),
bytes: wheelContents.byteLength,
},
},
null,
2,
)}\n`,
);
return { assetDir, wheelPath, manifestPath };
}
- Step 8: Install the runtime wheel in artifact smokes
Replace pythonArtifactInstallArgs with:
export function pythonArtifactInstallArgs(python, pythonArtifacts) {
return ['pip', 'install', '--python', python, pythonArtifacts.runtimeWheel];
}
Update pythonVerifySource() to assert kaelio-ktx metadata and keep module
imports:
export function pythonVerifySource() {
return `
import importlib.metadata
import semantic_layer
import ktx_daemon
assert importlib.metadata.version("kaelio-ktx") == "0.1.0"
assert semantic_layer is not None
assert ktx_daemon.PACKAGE_NAME == "ktx-daemon"
`;
}
- Step 9: Copy runtime assets before npm packing
Replace the loop in buildArtifacts(layout) with these explicit phases:
const commands = buildArtifactCommands(layout);
const npmBuildCount = NPM_ARTIFACT_PACKAGES.length;
const npmPackStart = commands.length - NPM_ARTIFACT_PACKAGES.length;
for (const command of commands.slice(0, npmBuildCount)) {
await runCommand(command.command, command.args, { cwd: command.cwd });
}
for (const command of commands.slice(npmBuildCount, npmPackStart)) {
await runCommand(command.command, command.args, { cwd: command.cwd });
}
const pythonArtifacts = await findPythonArtifacts(layout.pythonDir);
await copyRuntimeWheelAssets(layout, pythonArtifacts);
for (const command of commands.slice(npmPackStart)) {
await runCommand(command.command, command.args, { cwd: command.cwd });
}
- Step 10: Run package artifact tests
Run:
node --test scripts/package-artifacts.test.mjs
Expected: PASS.
- Step 11: Commit artifact wiring
Run:
git add scripts/package-artifacts.mjs scripts/package-artifacts.test.mjs
git commit -m "build: bundle python runtime wheel in cli artifacts"
Task 6: Update release policy and generated asset ignores
Files:
-
Modify:
release-policy.json -
Modify:
.gitignore -
Modify:
scripts/release-readiness.test.mjs -
Test:
scripts/release-readiness.test.mjs -
Step 1: Ignore generated CLI Python assets
Add this block to .gitignore after the dist/ ignore:
packages/cli/assets/python/
- Step 2: Add runtime wheel to release policy
Update release-policy.json so the Python packages list is:
"python": {
"publish": false,
"repository": null,
"packages": ["ktx-sl", "ktx-daemon", "kaelio-ktx"]
},
- Step 3: Update release readiness fixtures
In scripts/release-readiness.test.mjs, update fixture policy objects that
list Python packages from:
packages: ['ktx-sl', 'ktx-daemon'],
to:
packages: ['ktx-sl', 'ktx-daemon', 'kaelio-ktx'],
Update expected package name arrays to include kaelio-ktx:
packageNames: [
...NPM_ARTIFACT_PACKAGES.map((packageInfo) => packageInfo.name),
'ktx-sl',
'ktx-daemon',
'kaelio-ktx',
],
- Step 4: Run release readiness tests
Run:
node --test scripts/release-readiness.test.mjs
Expected: PASS.
- Step 5: Commit policy updates
Run:
git add .gitignore release-policy.json scripts/release-readiness.test.mjs
git commit -m "build: track bundled python runtime release artifact"
Task 7: Verify the built runtime wheel end to end
Files:
-
Build output:
dist/artifacts/python/kaelio_ktx-0.1.0-py3-none-any.whl -
Build output:
packages/cli/assets/python/manifest.json -
Build output:
packages/cli/assets/python/kaelio_ktx-0.1.0-py3-none-any.whl -
Step 1: Run focused script tests
Run:
node --test scripts/build-python-runtime-wheel.test.mjs scripts/package-artifacts.test.mjs scripts/release-readiness.test.mjs
Expected: PASS.
- Step 2: Run Python package tests affected by dependency split
Run:
uv run pytest python/ktx-daemon/tests -q
Expected: PASS.
- Step 3: Run package artifact check
Run:
pnpm run artifacts:check
Expected: PASS. This command builds the runtime wheel, copies it into CLI
assets before npm packing, installs the packed npm packages in a clean smoke
project, installs the bundled runtime wheel with uv pip install, and verifies
semantic_layer plus ktx_daemon imports from the one kaelio-ktx wheel.
- Step 4: Inspect the generated CLI asset manifest
Run:
node -e "const fs=require('node:fs'); const m=JSON.parse(fs.readFileSync('packages/cli/assets/python/manifest.json','utf8')); console.log(m.distributionName, m.version, m.wheel.file, m.wheel.sha256.length)"
Expected output:
kaelio-ktx 0.1.0 kaelio_ktx-0.1.0-py3-none-any.whl 64
- Step 5: Run pre-commit when configured
Run this only if .pre-commit-config.yaml exists:
uv run pre-commit run --files python/ktx-daemon/pyproject.toml uv.lock pyproject.toml scripts/build-python-runtime-wheel.mjs scripts/build-python-runtime-wheel.test.mjs scripts/package-artifacts.mjs scripts/package-artifacts.test.mjs scripts/release-readiness.test.mjs release-policy.json .gitignore
Expected: PASS. If no pre-commit config exists, record that no pre-commit configuration exists in this repository and skip this command.
- Step 6: Commit verification-only updates if any
If verification required small code or test fixes, commit them:
git add scripts/build-python-runtime-wheel.mjs scripts/build-python-runtime-wheel.test.mjs scripts/package-artifacts.mjs scripts/package-artifacts.test.mjs scripts/release-readiness.test.mjs python/ktx-daemon/pyproject.toml uv.lock release-policy.json .gitignore
git commit -m "test: verify bundled python runtime wheel"
If no files changed after verification, do not create an empty commit.
Acceptance criteria
dist/artifacts/python/kaelio_ktx-0.1.0-py3-none-any.whlis built bypnpm run artifacts:check.- The built CLI npm tarball includes
assets/python/kaelio_ktx-0.1.0-py3-none-any.whlandassets/python/manifest.json. - The asset manifest records the wheel filename, byte count, and SHA-256.
- Installing only the bundled runtime wheel exposes
semantic_layer,ktx_daemon, and thektx-daemonconsole script. sentence-transformersandtorchare absent from default dependencies and present under thelocal-embeddingsextra.- Existing separate
ktx-slandktx-daemonartifacts can remain CI artifacts in this plan; the npm runtime payload useskaelio-ktx.
Self-review
Spec coverage:
- Covers the package-model requirement for one bundled KTX-owned Python wheel.
- Covers the wheel checksum or runtime manifest requirement by adding the npm asset manifest.
- Covers lazy local embedding dependencies by moving heavy packages into the
local-embeddingsextra. - Leaves managed runtime directories, install commands, daemon reuse, and
@kaelio/ktxnpm renaming for later plans.
Placeholder scan:
- The plan contains no placeholder markers and no unspecified implementation steps.
Type and name consistency:
- Runtime distribution name is consistently
kaelio-ktx. - Wheel filename prefix is consistently
kaelio_ktx. - Runtime version is consistently
0.1.0.