Commit graph

44 commits

Author SHA1 Message Date
Luca Martial
5cf2c89093
Revise CLI reference docs (#100)
* docs: revise CLI reference

* docs: sync CLI reference with current commands
2026-05-14 12:53:55 -04:00
Andrey Avtomonov
ce23aca4c4
fix: remove project from ktx config (#95) 2026-05-14 17:39:31 +02:00
Andrey Avtomonov
b3be54e3fa
refactor(context): validate ktx.yaml with Zod and surface issues in status (#91)
* refactor(context): validate ktx.yaml with Zod and surface issues in status

- Replace hand-rolled ktx.yaml parsing with a strict Zod schema and
  derive KtxProjectConfig types from it.
- Add validateKtxProjectConfig returning structured KtxConfigIssue[]
  with migration hints for deprecated keys (ingest.llm,
  scan.enrichment.backend, etc.).
- Wire ktx status/doctor to run validation, render schema issues in
  plain and JSON output, and add a Config row to project status.
- Update the orbit example to camelCase scan.relationships keys to
  match the schema.

* fix(context): tolerate legacy setup.completed_steps and optional driver

- Accept and drop the legacy setup.completed_steps field so existing
  ktx.yaml files migrated from older versions still load.
- Make connections.<id>.driver optional in the schema; runtime code
  already produces a clear "no driver" error at use time.

* feat(cli): add ktx status --validate to run only ktx.yaml schema validation

- New --validate flag dispatches a focused runKtxDoctor 'validate' branch
  that reads ktx.yaml, runs validateKtxProjectConfig, and skips LLM,
  connection, embedding, and query-history checks.
- Plain output prints a single Config row; JSON output emits
  {ok: true} on success or the existing invalid_config / missing_project
  shapes on failure.
2026-05-14 15:36:35 +02:00
Andrey Avtomonov
b00c1a11a9
feat: merge ingest and scan
* docs: add CLI component reuse guidance

* docs: add unified ingest ux design

* Refine unified ingest UX design after adversarial review iteration 1

* Refine unified ingest UX design after adversarial review iteration 2

* Refine unified ingest UX design after adversarial review iteration 3

* feat(cli): route public connection ingest command

* feat(cli): hide standalone scan from public help

* feat(cli): plan public ingest depth and query history

* feat(cli): execute public database ingest facets

* feat(ingest): read connection query history config

* fix(cli): use public ingest wording

* fix(config): stop generating ingest adapter allow lists

* docs: document public ingest command

* test: align ingest surface expectations

* docs: add unified ingest public CLI surface plan

* feat(cli): preflight deep public ingest readiness

* feat(setup): store query history in connection context

* feat(setup): store database context depth

* feat(setup): verify context readiness by database depth

* fix(setup): keep context build foreground only

* fix(config): reject reserved ingest connection ids

* test: close unified ingest v1 expectations

* docs: add unified ingest v1 closure plan

* fix(ingest): bypass adapter allow-list for public source ingest

* fix(ingest): honor query history window intent

* fix(ingest): hide scan internals from public database ingest

* feat(ingest): use foreground view for interactive public ingest

* fix(setup): use schema context and query history wording

* test(cli): verify unified ingest public output

* docs: add unified ingest v1 public output closure plan

* fix(setup): forward query history flags

* fix(setup): prompt for postgres query history

* fix(status): report query history readiness

* fix(ingest): remove legacy public guidance

* fix(ingest): polish foreground retry copy

* docs(examples): use unified query history wording

* chore(ingest): finish public query history cleanup

* docs: add unified ingest v1 query history status cleanup plan

* test(docs): cover unified ingest public docs

* docs: align ingest CLI reference with unified UX

* docs: update context build guides for unified ingest

* docs: update setup and primary source ingest wording

* docs: stop advertising adapter-backed example ingest

* docs: close unified ingest public docs gaps

* docs: add unified ingest v1 docs site closure plan

* fix: render unified ingest foreground warnings

* fix: explain query history schema order

* fix: add public ingest retry guidance

* fix: align setup next steps with unified ingest

* fix: remove scan wording from demo progress

* test: verify unified ingest ux closure

* docs: add unified ingest v1 foreground and retry closure plan

* fix(cli): preserve query-history pull config in public ingest

* fix(cli): omit hidden commands from docs command tree

* test(cli): close unified ingest final public surface checks

* docs: add unified ingest v1 final public surface closure plan

* fix(cli): use public source labels in ingest reports

* fix(cli): suppress low-level public ingest output

* test(cli): verify unified ingest public plain output

* docs: add unified ingest v1 public plain output closure plan

* fix(cli): add public ingest copy sanitizers

* fix(cli): sanitize public ingest progress copy

* fix(cli): rename setup schema scope prompt

* docs(plan): add progress copy closure; test: align setup back-nav fixture

Adds the iter9 plan and updates the setup back-navigation test fixture
to pass disableQueryHistory plus listSchemas/listTables stubs that the
unified ingest setup step now requires.

* docs(plan): add final ux labels plan with narrowed label scans

* fix(cli): aggregate unsupported query-history warnings

* fix(cli): align setup database labels

* test(cli): fix setup database test type-check

* fix(cli): remove primary-source wording from setup output

* test(cli): verify unified ingest setup closure

* docs(plan): add unified ingest v1 verification copy closure plan

* fix(cli): remove top-level scan command

* fix(cli): remove legacy ingest and wiki commands

* Merge scan into ingest flow

* feat(cli): split ingest progress into per-phase rows, rename work units to tasks

Each database target in the unified ingest dashboard now renders one row per
real subprocess (Schema, then Query history when enabled) instead of a single
combined bar. Each phase has its own monotonic 0-100% bar so the progress
never snaps back to zero when historic-sql starts after scan completes.
Completed phases keep their final bar, summary, and elapsed time visible as
an inline audit trail; queued and skipped phases are shown explicitly.

Also rename user-facing "work units" / "Failed work units" to "tasks" /
"Failed tasks" in ingest output and parseIngestSummary. The parser still
accepts the legacy "Work units:" wording in captured output for backward
compat. Internal memory-flow event names and type fields are left alone.

* Fix test harness failures

* Fix CI smoke checks

---------

Co-authored-by: Andrey Avtomonov <7889985+andreybavt@users.noreply.github.com>
2026-05-14 01:43:06 +02:00
Andrey Avtomonov
1a472cf3ed
fix: clean up ktx yaml config parameters (#75)
* fix: clean up ktx yaml config parameters

* fix: align ci smoke checks with status output

* test: update artifact smoke status assertion
2026-05-14 01:27:31 +02:00
Andrey Avtomonov
0a261fe8a4
ci: add codecov coverage reporting (#82)
* ci: add codecov coverage reporting

* ci: fix codecov and secret scan checks

* ci: fix smoke and artifact checks
2026-05-14 01:13:31 +02:00
Andrey Avtomonov
fa9237956e
ci: run pre-commit checks in CI (#74)
* ci: run pre-commit in CI

* test: update CI workflow guardrail
2026-05-13 19:49:25 +02:00
Andrey Avtomonov
3fde4438b1
fix: stop requiring readonly connection config (#71) 2026-05-13 19:37:25 +02:00
Luca Martial
e50fef851f fix(cli): hide setup project banner 2026-05-13 09:16:35 -07:00
Andrey Avtomonov
d7147f9ca1
feat: rename project wiki directory (#66)
* feat: rename project wiki directory

* test: fix wiki skill ordering expectations

* Show configured context sources in setup
2026-05-13 16:05:58 +02:00
Andrey Avtomonov
97da9919e9
refactor: remove legacy compatibility paths (#64)
* refactor: remove legacy compatibility paths

* fix: support legacy metabase native queries

* test: use canonical semantic layer descriptions

* Rename CLI description

* Recover setup scan from SQLite ABI mismatch

* Remove legacy product name from CLI help
2026-05-13 15:55:00 +02:00
Andrey Avtomonov
c202202e6b
feat(cli): clean up wiki and sl commands (#65)
* feat(cli): clean up wiki and sl commands

* test(scripts): update package artifact CLI smoke assertion
2026-05-13 15:41:10 +02:00
Andrey Avtomonov
bcb0d2f8f7
chore: add TypeScript dead-code checks (#60)
* chore: add TypeScript dead-code checks

* chore: trim stale Knip ignores

* Fix CI smoke and artifact checks
2026-05-13 13:33:28 +02:00
Andrey Avtomonov
721f1a998f
feat(cli)!: remove ktx agent command (#58)
* feat(cli)!: remove ktx agent command

* test(context): update PGlite boundary guardrail
2026-05-13 13:01:56 +02:00
Andrey Avtomonov
eaaabb361e
fix(cli): clean up dev runtime commands (#59) 2026-05-13 12:28:24 +02:00
Andrey Avtomonov
b9e0a746af
feat(cli): clean up dev command surface (#57)
* feat(cli): clean up dev command surface

* test: align CI expectations with CLI cleanup

* test(cli): update slow test command expectations
2026-05-13 12:00:08 +02:00
Andrey Avtomonov
85fc408054
chore(deps): refresh workspace dependencies (#43)
* chore(deps): refresh workspace dependencies

* Fix pnpm artifact smoke build approvals
2026-05-13 01:15:35 +02:00
Andrey Avtomonov
17a2fee69a
fix(cli): remove ktx setup subcommands (#42)
* fix(cli): remove ktx setup subcommands

* test(scripts): update setup-dev status expectation
2026-05-13 00:38:26 +02:00
Andrey Avtomonov
e15a4ebaec feat(cli): clean up command surface 2026-05-12 23:51:46 +02:00
Luca Martial
60457e9407
Improve schema setup and Notion ingest UX (#14)
* Improve schema setup and Notion ingest UX

* Handle Postgres network scan failures

* WIP: save local changes before main merge

* Refine setup prompt choices

* Tighten ingest reconciliation guidance

* Commit setup config updates

* Canonicalize unmapped fallback details

* Count reconciliation actions in reports

* Harden semantic layer source validation

* Return wiki content after edits

* Validate SL sources against manifests

* Validate wiki refs before writes

* Simplify CLI next steps

* Clarify agent setup summary

* Surface dbt target SL sources

* Recover SL write fallbacks

* Preserve failed context build metadata

* Track raw paths for ingest actions

* test(cli): update seeded demo expectations

* fix(ingest): scope fallback recovery checks

* fix(sl): tighten source validation guards

* fix(wiki): ignore empty embedding vectors

* Improve Notion ingest UX

* Enforce flat wiki keys

* test(context): update wiki key assertion

---------

Co-authored-by: Andrey Avtomonov <andreybavt@gmail.com>
2026-05-12 22:56:58 +02:00
Andrey Avtomonov
5277c81b33 fix(ci): update artifact source test 2026-05-12 16:17:45 +02:00
Andrey Avtomonov
6a1fded5ce fix(ci): align smoke stderr expectations 2026-05-12 15:31:41 +02:00
Andrey Avtomonov
15f433930e
Merge branch 'main' into andreybavt/execute-context7-plan 2026-05-12 13:04:16 +02:00
Andrey Avtomonov
52400c599c chore: standardize pre-commit checks 2026-05-12 13:02:06 +02:00
Andrey Avtomonov
bd5154f918
Merge pull request #31 from Kaelio/andreybavt/pasted-text-attachment
fix: standardize KTX environment variables
2026-05-12 12:26:33 +02:00
Andrey Avtomonov
4c93a6e983
fix(ci): update stale KTX test expectations (#32)
Co-authored-by: Andrey Avtomonov <7889985+andreybavt@users.noreply.github.com>
2026-05-12 12:02:26 +02:00
Andrey Avtomonov
d5f484eb7e fix: standardize KTX environment variables 2026-05-12 11:21:37 +02:00
Andrey Avtomonov
bb8868f238
Merge pull request #19 from Kaelio/andreybavt/dbt-vertex-no-anthropic
fix(cli): honor configured LLM backends in setup
2026-05-12 10:25:24 +02:00
Andrey Avtomonov
3e9869340f ci: parallelize KTX CI checks 2026-05-12 01:44:15 +02:00
Andrey Avtomonov
0153ac4945 Relax boundary check for test fixtures 2026-05-12 01:34:18 +02:00
Andrey Avtomonov
f3f6b36551 Merge remote-tracking branch 'origin/main' into andreybavt/historic-sql-redesign 2026-05-11 20:52:19 +02:00
Andrey Avtomonov
ca325cd72c test: verify historic sql sharded smoke docs 2026-05-11 20:32:32 +02:00
Andrey Avtomonov
194752756a test: expect historic sql pattern shard smoke docs 2026-05-11 20:30:14 +02:00
Andrey Avtomonov
69128ccf72 fix: link Conductor agent overlays from root checkout 2026-05-11 20:06:13 +02:00
Andrey Avtomonov
81ec2eee7c test: verify historic sql docs and smoke cleanup 2026-05-11 19:42:51 +02:00
Andrey Avtomonov
598149b6a4 test: expect unified historic sql example docs 2026-05-11 19:39:21 +02:00
Andrey Avtomonov
a46563bb01 chore: move docs site workspace 2026-05-11 16:53:42 +02:00
Andrey Avtomonov
0ae9b6effd
Expose .agents/skills overlay to Claude Code (#10)
Conductor workspaces symlink `.agents` from KTX_AGENT_OVERLAYS_ROOT, which
Codex CLI reads directly. Claude Code only scans `.claude/skills/`, so the
shared skills were invisible to it. Add `link_agent_skills_for_claude` to
mirror each `.agents/skills/*` entry as a symlink under `.claude/skills/*`.
Idempotent and refuses to clobber existing non-matching entries.
2026-05-11 16:21:40 +02:00
Andrey Avtomonov
9dad936ac7
feat: npm-managed Python runtime for @kaelio/ktx (#7)
* docs: add npm managed python runtime design

* build: add bundled python runtime wheel builder

* build: make local embedding dependencies optional

* build: bundle python runtime wheel in cli artifacts

* build: track bundled python runtime release artifact

* test: verify bundled python runtime wheel

* docs: add plan for bundled python runtime wheel

* test: cover managed python runtime lifecycle

* feat: add managed python runtime installer

* feat: add runtime command runner

* feat: expose runtime management commands

* test: verify managed python runtime commands

* docs: add plan for managed python runtime installer

* feat: add managed python command helper

* feat: use managed runtime for sl query compute

* feat: route sl query managed runtime policy

* docs: add plan for managed runtime sl query integration

* feat: add managed runtime daemon metadata

* feat: manage python daemon lifecycle

* feat: add runtime daemon start stop commands

* fix: verify managed runtime daemon lifecycle

* docs: add plan for managed runtime daemon lifecycle

* feat: add managed local embeddings config marker

* feat: add managed local embeddings daemon helper

* feat: use managed runtime for local embedding setup

* feat: pass managed runtime policy through setup

* docs: add plan for managed local embeddings runtime

* feat: read CLI package metadata dynamically

* feat: assemble public kaelio ktx npm package

* feat: release one public kaelio ktx npm artifact

* test: cover public kaelio ktx package invocations

* chore: verify public kaelio ktx package artifacts

* docs: add plan for public kaelio ktx npm package

* test: verify managed runtime in public package smoke

* test: finalize managed runtime release smoke

* docs: add plan for managed runtime release smoke

* test: specify local embeddings release smoke

* feat: add local embeddings runtime smoke

* chore: register local embeddings smoke

* fix: verify local embeddings smoke

* fix: restore artifact smoke python env helper

* docs: add plan for managed local embeddings release smoke

* refactor: share managed runtime install policy parsing

* feat: use managed runtime for agent semantic queries

* feat: use managed runtime for MCP semantic compute

* docs: add plan for managed agent and MCP semantic runtime

* feat(cli): add managed daemon HTTP helpers

* feat(cli): route local adapters through managed daemon

* feat(cli): use managed daemon for ingest helpers

* feat(cli): pass managed daemon options to scan

* feat(context): pass MCP ingest pull config options

* feat(cli): pass managed daemon options to serve ingest

* test: verify managed local ingest daemon runtime

* docs: add plan for managed local ingest daemon runtime

* docs: align managed runtime examples

* docs: add plan for managed runtime docs cleanup

* test: cover published package runtime smoke commands

* test: validate published package smoke outputs

* docs: add plan for published package runtime smoke

* build: stamp public npm package version

* release: add npm public release policy

* release: add guarded npm publish script

* release: document public npm release handoff

* docs: add plan for public npm release handoff

* test: cover managed runtime prune in package smoke

* docs: document managed runtime prune

* docs: add plan for managed runtime prune smoke and docs

* chore: encode uv runtime prerequisite policy

* fix: clarify missing uv runtime error

* docs: document uv runtime prerequisite

* docs: add plan for uv runtime prerequisite contract

* refactor: limit release artifacts to public package runtime

* chore: align release policy with bundled runtime wheel

* docs: describe single public runtime artifact surface

* test: verify single public runtime artifact contract

* docs: add plan for single public runtime artifact cleanup

* fix: align local embeddings smoke with public version

* docs: add plan for local embeddings smoke public version

* release: soft-launch as @kaelio/ktx@0.1.0-rc.0 on next tag

Publish target moves to the pre-release version 0.1.0-rc.0 under the next
dist-tag so npm install @kaelio/ktx (which resolves to latest) does not
pick up the soft-launch build. Users opt in via @kaelio/ktx@next.

* Fix release script boundary checks

* Remove PostHog from public package bundle
2026-05-11 15:50:34 +02:00
Andrey Avtomonov
075764fe77
[codex] Add Conductor workspace setup (#5)
* Add Conductor workspace setup

* Refine Conductor setup script structure

* Separate Conductor setup steps

* Consolidate Conductor setup script

* Use KTX env var for agent overlays

* Restore Orbit verification summary docs
2026-05-11 10:40:30 +02:00
Andrey Avtomonov
76fde89798
[codex] Add Conductor workspace scripts (#2)
* Add Conductor workspace scripts

* Fix conductor boundary check fixture

* Remove stale frontend conductor guard

* Remove stale app service references

* Optimize relationship discovery benchmarks

* test: move slow suites to ci tier
2026-05-11 09:55:42 +02:00
Luca Martial
b3dcb577d9 misc 2026-05-10 20:44:07 -07:00
Andrey Avtomonov
3ce510b55b rename klo to ktx 2026-05-10 23:51:24 +02:00
Andrey Avtomonov
1a42152e6f Initial open-source release 2026-05-10 23:12:26 +02:00