Commit graph

17 commits

Author SHA1 Message Date
Andrey Avtomonov
c1ed5eedce
fix(cli): preserve project artifacts when ktx setup steps fail (#229)
ktx setup wiped ktx.yaml, .ktx/setup/state.json, wiki/, semantic-layer/,
raw-sources/, and .git/ — or removed the entire project dir — whenever any
single source in the context-build step failed, destroying hours of ingest
work and the persisted resume state. The cleanup hint was designed for an
"early abort, leave no trace" semantic but was applied indiscriminately to
every later step failure, in direct conflict with the .ktx/setup/state.json
resume mechanism.

Drop the cleanup mechanism entirely (KtxSetupCreatedProjectCleanup,
cleanupForFolderState, createProjectWithCleanup, cleanupCreatedProjectScaffold,
and the createdProjectCleanup plumbing through KtxSetupProjectResult). Step
failures now return non-zero without touching the filesystem, so re-running
ktx setup continues from completed steps and only re-attempts failed sources.

Rewrites the two tests that documented the wipe behavior to assert
preservation, and adds a regression test that simulates partial context-build
artifacts (state.json, wiki/, semantic-layer/) and verifies all survive a
failed context step.

Refs KLO-719
2026-05-28 15:17:06 +02:00
Andrey Avtomonov
2366b00301
chore(workspace): gate dead-code with knip production mode (#196)
* refactor(workspace): relocate @ktx/llm source into packages/cli/src/llm

* refactor(workspace): rewrite @ktx/llm imports to relative paths

* refactor(workspace): fold internal packages into cli

* chore(workspace): gate dead-code with knip production mode

Turn on production-mode knip plus an autofix run in pre-commit and the
`pnpm dead-code` script, document the `/** @internal */` convention for
test-only exports in AGENTS.md, annotate test-only exports across the
CLI with that JSDoc, and drop dead exports/wrappers the new gate
surfaced (e.g. `cli-project.ts`, `lookerRuntimeSourceToFileAdapterSource`,
`createLocalScanEnrichmentProvidersFromConfig`,
`PGLITE_OWNER_PROCESS_BACKEND_CAPABILITIES`, stale type re-exports).
Replace the loose `ignoreIssues` allowlist in `knip.json` with explicit
production entries so cross-package barrel leaks are caught.

* refactor(cli): delete internal barrel index.ts files

The 34 `index.ts` re-export barrels inside `packages/cli/src/` were
holdovers from the pre-fold multi-workspace structure. Post-fold-in they
served no production purpose: external consumers go through the single
package main entry, and in-repo callers mostly imported through them
only because the path was short. Internally, knip flagged most barrel
re-exports as production-dead (only reached via tests).

This change:
- Deletes every internal barrel except `packages/cli/src/index.ts`
  (the published package entry).
- Rewrites ~270 source/test files to import each name directly from
  the file that defines it.
- Moves `tools/warehouse-verification/index.ts` to
  `create-warehouse-verification-tools.ts` (the function it defined
  locally) and updates its single consumer.
- Renames `search/backend-conformance.ts` → `.test-utils.ts` to match
  the existing test-helper file convention.
- Deletes 13 dead test-only chains (dbt-descriptions/*,
  live-database/extracted-schema, live-database/structural-sync,
  relationship-* feedback/review chain) plus their tests and a
  cascading orphan integration test.
- Updates test mocks that pointed at deleted barrel paths
  (notion-client, connector barrels in scan/local-scan-connectors
  tests) to mock the source files instead.
- Points the maintainer benchmark script
  (`scripts/relationship-benchmark-report.mjs`) at source files
  instead of `dist/context/scan/index.js`.
- Drops the barrel `!` entries from `knip.json`; adds explicit
  production entries only for the benchmark code reached via dist by
  the maintainer script.

Net: 413 files changed, ~1.2k insertions, ~9.4k deletions.

`pnpm run dead-code` (Biome + knip default + knip production) and
`pnpm run type-check` are clean; 2277 tests pass.

* refactor(workspace): rename @ktx/cli to @kaelio/ktx and pack it directly

Promote the CLI workspace package to the public name `@kaelio/ktx` and
drop the separate `scripts/build-public-npm-package.mjs` wrapper. The
CLI package is now publishable in place (`publishConfig.access: public`,
`provenance: true`), so artifact packing uses `pnpm pack` against
`packages/cli/` instead of assembling a parallel package tree.

Updates all workspace filter invocations, docs, tests, and release
readiness checks to reference the new package name, and folds the
tarball-name helper into `scripts/public-npm-release-metadata.mjs`.

* docs: align "agent clients" and "data agents" terminology

Replace "client agents" with "agent clients" and "database agents" with
"data agents" across AGENTS.md, README.md, the docs-site copy, and the
matching setup-agents test description, matching the canonical
vocabulary in docs/terminology.md.

Also moves packages/cli/tsconfig.json's tsBuildInfoFile from
node_modules/.cache/ to dist/.tsbuildinfo so incremental builds survive
node_modules reinstalls.

* refactor(release): single source of truth for package version

Make packages/cli/package.json the single source of truth for the
@kaelio/ktx version. publicNpmPackageVersion() now reads it directly,
so artifact filenames, release-readiness checks, and the Python wheel
version all derive from one field. The duplicate
release-policy.json.publicNpmPackageVersion is removed.

Previously the two fields could drift: tarballs were named
kaelio-ktx-0.4.1.tgz while internally containing
@kaelio/ktx@0.0.0-private.

- update-public-release-version.mjs rewrites both Python pyproject.toml
  files (ktx-daemon, ktx-sl) alongside the npm package.jsons,
  normalizing the version for PEP 440 (e.g. 0.1.0-rc.2 -> 0.1.0rc2).
- semantic-release-config.cjs adds the two pyproject.toml files to
  @semantic-release/git assets so the release commit back to main
  carries every version source in lockstep.
- The six "?? '0.0.0-private'" fallback literals across the CLI are
  replaced with "?? getKtxCliPackageInfo().version", and
  createDefaultKtxMcpServer makes its version arg required.
- docs/release.md describes the actual commit-back model: the dev tree
  always reflects the most recent release; no sentinel pin to
  maintain.

Verified: pnpm run artifacts:build now produces
kaelio-ktx-0.4.1.tgz and kaelio_ktx-0.4.1-py3-none-any.whl with
@kaelio/ktx@0.4.1 inside. Full type-check, dead-code, and
2287 vitests + 173 script tests pass.

* refactor(cli): inject embedding provider resolution and detect sentence-transformers runtime

Make resolveProjectEmbeddingProvider and runtimeIo injectable in ingest and
scan command entrypoints so tests can stub them, and teach
resolvePublicIngestRuntimeRequirements to flag the local-embeddings runtime
feature when ktx.yaml selects sentence-transformers.

* chore(cli): mark buildLocalStatsStatus and LocalStatsStatus as @internal

Both symbols are consumed only by status-project.test.ts. Annotating with
/** @internal */ keeps knip's production-mode check clean without changing
runtime behavior.

* fix(cli): use real package metadata in print-command-tree

The stubbed package name embedded a forbidden product identifier that
tripped the boundary check in CI. Read the metadata from package.json
instead — keeps the rendered tree unchanged and removes a duplicate
source of truth.

* feat(cli): show embedding coverage in `ktx status`, drop duplicate disk counts

Inline `(N embedded)` next to the Wiki scope counts and Semantic-layer
source counts, computed with `SUM(embedding_json IS NOT NULL)` over
`knowledge_pages` and `local_sl_sources`. Rename the "Knowledge" label to
"Wiki" (canonical per `docs/terminology.md`) and rename the matching
`localStats.knowledgePages` field to `localStats.wikiPages`.

Drop `wiki=N md` and `semantic-layer=N yaml` from the Disk row — those
duplicated the per-surface rows above. Disk now reports only actual byte
usage (db, cache, raw-sources). The unused `wikiGlobalMarkdownCount` /
`semanticLayerYamlCount` fields, the `isMarkdownEntry` / `isYamlEntry`
helpers, and the `filter` arg on `summarizeDir` are removed.
2026-05-21 15:28:58 +02:00
Andrey Avtomonov
590dd5dddb
fix(cli): simplify setup flags and agents tty handling (#155)
* fix(cli): simplify setup flags and agents tty handling

* fix(context): update ingest setup guidance flag
2026-05-19 19:23:35 +02:00
Andrey Avtomonov
af0567c57e
fix(cli): install managed runtime with required Python (#152) 2026-05-19 18:18:38 +02:00
Andrey Avtomonov
ce23aca4c4
fix: remove project from ktx config (#95) 2026-05-14 17:39:31 +02:00
Andrey Avtomonov
754e4a9039
feat(cli): improve setup progress UX (#69) 2026-05-13 17:01:48 +02:00
Andrey Avtomonov
d4d8ad1724
fix: show setup destination paths (#63) 2026-05-13 14:27:29 +02:00
Andrey Avtomonov
b75576279c
fix: store Metabase mappings in ktx.yaml (#61)
* fix: store Metabase mappings in ktx.yaml

* docs: note KTX has no public users

* refactor: drop setup progress compatibility
2026-05-13 13:55:21 +02:00
Luca Martial
52d1c903ae fix(cli): remove redundant path resolution hints from project setup
The placeholder text and confirmation step already communicate path
formats clearly; the standalone hints added visual noise.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 17:17:18 -07:00
Luca Martial
ea10dd9f39 merge: resolve conflict with main's box-drawing formatting
Keep the confirmProjectDir helper extraction from this branch while
adopting the │ box-drawing prefixes added on main.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 17:03:03 -07:00
Luca Martial
bdca6d0f04 fix(cli): replace duplicate directory prompt with direct path options
Extract confirmProjectDir helper and split the "Create a new project
folder" option into "New subfolder (./ktx-project)" and "Custom path"
so users reach their target directory with fewer prompts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 16:59:30 -07:00
Luca Martial
fcdf5234c6
Merge pull request #45 from Kaelio/luca/klo-654-improve-indents
feat(cli): add box-drawing prefixes to setup messages
2026-05-12 19:58:55 -04:00
Luca Martial
07ac71ea7c feat(cli): add box-drawing prefixes to remaining setup stdout messages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 16:58:09 -07:00
Luca Martial
dbfee6b453 feat(cli): migrate all setup steps to use local state for completion tracking
Update every setup step to write completed_steps to .ktx/setup/state.json
instead of ktx.yaml, stripping legacy entries from config on write.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 16:26:23 -07:00
Luca Martial
f091f948ee feat(cli): add box-drawing prefixes to setup informational messages
Align stdout informational messages in setup flows with the existing
Clack prompt visual style by prefixing them with │.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 15:46:56 -07:00
Andrey Avtomonov
3ce510b55b rename klo to ktx 2026-05-10 23:51:24 +02:00
Andrey Avtomonov
1a42152e6f Initial open-source release 2026-05-10 23:12:26 +02:00