feat: npm-managed Python runtime for @kaelio/ktx (#7)

* docs: add npm managed python runtime design

* build: add bundled python runtime wheel builder

* build: make local embedding dependencies optional

* build: bundle python runtime wheel in cli artifacts

* build: track bundled python runtime release artifact

* test: verify bundled python runtime wheel

* docs: add plan for bundled python runtime wheel

* test: cover managed python runtime lifecycle

* feat: add managed python runtime installer

* feat: add runtime command runner

* feat: expose runtime management commands

* test: verify managed python runtime commands

* docs: add plan for managed python runtime installer

* feat: add managed python command helper

* feat: use managed runtime for sl query compute

* feat: route sl query managed runtime policy

* docs: add plan for managed runtime sl query integration

* feat: add managed runtime daemon metadata

* feat: manage python daemon lifecycle

* feat: add runtime daemon start stop commands

* fix: verify managed runtime daemon lifecycle

* docs: add plan for managed runtime daemon lifecycle

* feat: add managed local embeddings config marker

* feat: add managed local embeddings daemon helper

* feat: use managed runtime for local embedding setup

* feat: pass managed runtime policy through setup

* docs: add plan for managed local embeddings runtime

* feat: read CLI package metadata dynamically

* feat: assemble public kaelio ktx npm package

* feat: release one public kaelio ktx npm artifact

* test: cover public kaelio ktx package invocations

* chore: verify public kaelio ktx package artifacts

* docs: add plan for public kaelio ktx npm package

* test: verify managed runtime in public package smoke

* test: finalize managed runtime release smoke

* docs: add plan for managed runtime release smoke

* test: specify local embeddings release smoke

* feat: add local embeddings runtime smoke

* chore: register local embeddings smoke

* fix: verify local embeddings smoke

* fix: restore artifact smoke python env helper

* docs: add plan for managed local embeddings release smoke

* refactor: share managed runtime install policy parsing

* feat: use managed runtime for agent semantic queries

* feat: use managed runtime for MCP semantic compute

* docs: add plan for managed agent and MCP semantic runtime

* feat(cli): add managed daemon HTTP helpers

* feat(cli): route local adapters through managed daemon

* feat(cli): use managed daemon for ingest helpers

* feat(cli): pass managed daemon options to scan

* feat(context): pass MCP ingest pull config options

* feat(cli): pass managed daemon options to serve ingest

* test: verify managed local ingest daemon runtime

* docs: add plan for managed local ingest daemon runtime

* docs: align managed runtime examples

* docs: add plan for managed runtime docs cleanup

* test: cover published package runtime smoke commands

* test: validate published package smoke outputs

* docs: add plan for published package runtime smoke

* build: stamp public npm package version

* release: add npm public release policy

* release: add guarded npm publish script

* release: document public npm release handoff

* docs: add plan for public npm release handoff

* test: cover managed runtime prune in package smoke

* docs: document managed runtime prune

* docs: add plan for managed runtime prune smoke and docs

* chore: encode uv runtime prerequisite policy

* fix: clarify missing uv runtime error

* docs: document uv runtime prerequisite

* docs: add plan for uv runtime prerequisite contract

* refactor: limit release artifacts to public package runtime

* chore: align release policy with bundled runtime wheel

* docs: describe single public runtime artifact surface

* test: verify single public runtime artifact contract

* docs: add plan for single public runtime artifact cleanup

* fix: align local embeddings smoke with public version

* docs: add plan for local embeddings smoke public version

* release: soft-launch as @kaelio/ktx@0.1.0-rc.0 on next tag

Publish target moves to the pre-release version 0.1.0-rc.0 under the next
dist-tag so npm install @kaelio/ktx (which resolves to latest) does not
pick up the soft-launch build. Users opt in via @kaelio/ktx@next.

* Fix release script boundary checks

* Remove PostHog from public package bundle
This commit is contained in:
Andrey Avtomonov 2026-05-11 15:50:34 +02:00 committed by GitHub
parent 075764fe77
commit 9dad936ac7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
99 changed files with 25375 additions and 1538 deletions

View file

@ -4,14 +4,21 @@ The package artifact smoke checks create temporary projects instead of storing
sample projects in this directory. Run the checks from `ktx/`:
```bash
source .venv/bin/activate
pnpm run artifacts:check
```
The npm smoke project installs the generated `@ktx/context` and `@ktx/cli`
tarballs, imports public package entry points, and runs installed `ktx`
commands against a generated local project.
The npm smoke project installs the generated public `@kaelio/ktx` tarball,
imports the package entry point, and runs installed `ktx` commands against a
generated local project.
The Python smoke project installs `ktx-daemon` through the local artifact
directory, imports `semantic_layer` and `ktx_daemon`, and runs
`python -m ktx_daemon semantic-validate`.
The managed Python runtime smoke requires `uv` on `PATH`, isolates
`KTX_RUNTIME_ROOT`, verifies `ktx runtime status`, runs `ktx sl query --yes` to
install the core runtime from the bundled wheel, checks `ktx runtime doctor`,
starts and reuses the managed daemon, stops it, previews a stale runtime with
`ktx runtime prune --dry-run`, verifies confirmation is required, and removes
the stale runtime with `ktx runtime prune --yes`.
The artifact manifest contains the public `@kaelio/ktx` npm tarball and the
bundled `kaelio-ktx` runtime wheel. The smoke does not install standalone
Python packages directly; Python-backed behavior is verified through the
managed runtime installed from the npm package.

View file

@ -13,8 +13,8 @@ generates query workload under separate users, runs `ktx setup` with
- Docker with Compose v2
- Node and pnpm matching the KTX workspace
- `KTX_SQL_ANALYSIS_URL` or `KTX_DAEMON_URL` pointing at a running SQL-analysis
service that exposes `/api/sql/analyze-for-fingerprint`
- `uv` on `PATH` so the KTX-managed Python runtime can install the bundled
runtime wheel
## Run
@ -24,8 +24,9 @@ From the KTX repository root:
examples/postgres-historic/scripts/smoke.sh
```
The smoke creates a temporary KTX project, starts Postgres on
`127.0.0.1:55432`, and uses this connection URL:
The smoke creates a temporary KTX project, isolates the managed Python runtime
under the temporary project parent, starts Postgres on `127.0.0.1:55432`, and
uses this connection URL:
```bash
postgresql://ktx_reader:ktx_reader@127.0.0.1:55432/analytics # pragma: allowlist secret
@ -83,10 +84,11 @@ Historic SQL (warehouse)` when `pg_stat_statements` is installed,
Run local historic-SQL ingest:
```bash
node packages/cli/dist/bin.js --project-dir /tmp/ktx-postgres-historic dev ingest run \
pnpm run ktx -- dev ingest run --project-dir /tmp/ktx-postgres-historic \
--connection-id warehouse \
--adapter historic-sql \
--plain \
--yes \
--no-input
```
@ -111,5 +113,6 @@ The manifest should have `dialect: "postgres"`, `degraded: true`,
- Missing grants: confirm `GRANT pg_read_all_stats TO ktx_reader;`.
- Empty templates: rerun `scripts/generate-workload.sh base` and keep
`--historic-sql-min-calls 2` for the smoke.
- SQL-analysis failures: set `KTX_SQL_ANALYSIS_URL` or `KTX_DAEMON_URL` to a
running service URL before running `scripts/smoke.sh`.
- SQL-analysis failures: run `pnpm run ktx -- runtime doctor` from the KTX
repository root and confirm `uv`, the bundled Python wheel, and the managed
runtime all pass.

View file

@ -8,22 +8,20 @@ COMPOSE_FILE="$EXAMPLE_DIR/docker-compose.yml"
PROJECT_PARENT="${KTX_POSTGRES_HISTORIC_PROJECT_PARENT:-$(mktemp -d)}"
PROJECT_DIR="$PROJECT_PARENT/postgres-historic-ktx"
KTX_BIN="$KTX_ROOT/packages/cli/dist/bin.js"
export KTX_RUNTIME_ROOT="$PROJECT_PARENT/managed-runtime"
unset KTX_DAEMON_URL
unset KTX_SQL_ANALYSIS_URL
cleanup() {
if [[ -f "$KTX_BIN" ]]; then
node "$KTX_BIN" runtime stop >/dev/null 2>&1 || true
fi
if [[ "${KTX_POSTGRES_HISTORIC_KEEP_DOCKER:-0}" != "1" ]]; then
docker compose -f "$COMPOSE_FILE" down -v >/dev/null 2>&1 || true
fi
}
trap cleanup EXIT
require_sql_analysis_url() {
if [[ -n "${KTX_SQL_ANALYSIS_URL:-}" || -n "${KTX_DAEMON_URL:-}" ]]; then
return
fi
echo "Set KTX_SQL_ANALYSIS_URL or KTX_DAEMON_URL before running this smoke." >&2
exit 1
}
latest_manifest() {
find "$PROJECT_DIR/raw-sources/warehouse/historic-sql" -name manifest.json | sort | tail -n 1
}
@ -60,9 +58,19 @@ const jobId = process.argv[4];
const { loadKtxProject } = await import(join(ktxRoot, 'packages/context/dist/project/index.js'));
const { runLocalStageOnlyIngest } = await import(join(ktxRoot, 'packages/context/dist/ingest/index.js'));
const { createKtxCliLocalIngestAdapters } = await import(join(ktxRoot, 'packages/cli/dist/local-adapters.js'));
const { getKtxCliPackageInfo } = await import(join(ktxRoot, 'packages/cli/dist/index.js'));
const project = await loadKtxProject({ projectDir });
const adapters = createKtxCliLocalIngestAdapters(project, { historicSqlConnectionId: 'warehouse' });
const cliVersion = getKtxCliPackageInfo().version;
const managedRuntimeIo = { stdout: process.stdout, stderr: process.stderr };
const adapters = createKtxCliLocalIngestAdapters(project, {
historicSqlConnectionId: 'warehouse',
managedDaemon: {
cliVersion,
installPolicy: 'auto',
io: managedRuntimeIo,
},
});
const adapter = adapters.find((candidate) => candidate.source === 'historic-sql');
if (!adapter) throw new Error('historic-sql adapter was not registered for local run');
const record = await runLocalStageOnlyIngest({
@ -88,7 +96,6 @@ NODE
cd "$KTX_ROOT"
pnpm --filter @ktx/context run build
pnpm --filter @ktx/cli run build
require_sql_analysis_url
docker compose -f "$COMPOSE_FILE" up -d --wait
"$EXAMPLE_DIR/scripts/generate-workload.sh" base