Commit graph

38 commits

Author SHA1 Message Date
Andrey Avtomonov
590dd5dddb
fix(cli): simplify setup flags and agents tty handling (#155)
* fix(cli): simplify setup flags and agents tty handling

* fix(context): update ingest setup guidance flag
2026-05-19 19:23:35 +02:00
Andrey Avtomonov
af0567c57e
fix(cli): install managed runtime with required Python (#152) 2026-05-19 18:18:38 +02:00
Andrey Avtomonov
06aeb56f39
fix: remove deterministic embedding backend (#146)
* fix: remove deterministic embedding backend

* test: update slow tests for disabled embeddings
2026-05-19 16:40:01 +02:00
Andrey Avtomonov
56f4f9c9e8
feat(cli): split Claude Desktop skills and polish setup-agents output (#141)
* fix(cli): package Claude Desktop skills in one zip

* Polish ktx setup-agents output: hint, summaries, outro

* test: update setup agents output polish assertion

* Add output-polish follow-up plan

* docs: align Claude Desktop split-ZIP wording

Update README and the agent-clients docs page to reflect that ktx setup
now produces one uploadable ZIP per Claude Desktop skill under
.ktx/agents/claude/ (ktx-analytics.zip and optionally ktx.zip) instead
of a single combined ktx-skills.zip.

* feat(cli): style next-actions note in TTY mode

Add createAgentNextActionsLineFormatter, an ANSI line transformer wired
into the "Required before using agents" Clack note. It activates only
when the target stream reports hasColors(), so non-TTY pipelines and
tests keep the existing plain-text output byte-identical.

Per-line rules: cyan-bold step numbers + bold titles; dim sub-prose
aligned under the title; dim-cyan bullet for .zip paths with HOME
shortened to ~; dim "›" replaces " > " breadcrumbs; RUN/PASTE/USE/OPEN
markers dimmed; already-styled lines pass through to avoid double-wrap.

* docs: move output polish specs out of ktx
2026-05-19 15:21:49 +02:00
Andrey Avtomonov
b42f418adc
fix: allow agent setup without context (#139)
* fix: allow agent setup without context

* docs: align readme command examples
2026-05-19 12:18:52 +02:00
Luca Martial
1331e573dd
Improve KTX agent setup guidance (#137)
* feat(cli): clarify MCP start output

* feat(cli): improve agent setup guidance

* docs: update agent client setup guidance
2026-05-18 18:54:20 -04:00
Andrey Avtomonov
c89af7733a
fix: improve ingest runtime readiness (#124)
* fix: improve ingest runtime readiness

* fix(cli): mock runtime in slow setup tests

* test(cli): isolate setup runtime status
2026-05-17 10:27:29 +02:00
Andrey Avtomonov
e6d578c03f
feat(setup): add Claude Desktop target and MCP-first agent setup (#114)
* feat(setup): add Claude Desktop target and MCP-first agent setup

Adds `ktx mcp stdio` and a `claude-desktop` setup target that generates a
local plugin ZIP wiring the analytics skill and a stdio MCP config. Replaces
the CLI-only agent install mode with MCP+analytics (default) and an optional
admin CLI skill, renames the research skill to analytics, and lets interactive
setup pick project vs global scope when every target supports it. Extracts a
shared MCP server factory used by both HTTP and stdio entrypoints.

* Add MCP agent client setup support

* Polish setup output formatting

* Add MCP tool polish design spec

Design for slimming the MCP-registered surface from 25 to 11 tools,
introducing memory_ingest, applying the per-tool polish kit (annotations,
outputSchema, .describe(), in-band error wrapping, union-drift fixes,
type-narrowed jsonToolResult), emitting progress notifications on
sql_execution + sl_query, and refining the ktx-analytics SKILL.md to
match.

* Refine MCP tool polish design spec after adversarial review iteration 1

* Refine MCP tool polish design spec after adversarial review iteration 2

* Refine MCP tool polish design spec after adversarial review iteration 3

* refactor(context): rename memory capture service to ingest

* feat(mcp): slim research tool surface

* refactor(mcp): remove admin ports from server factory

* refactor(cli): rename text ingest memory port

* docs: update analytics skill for memory ingest

* chore: verify mcp surface rename

* Add MCP tool polish v1 surface change plan

* feat(context): polish mcp tool metadata

* fix(context): enforce resolved semantic layer compute sources

* feat(context): emit mcp query progress stages

* fix(context): keep mcp progress event internal

* Add MCP tool polish v1 metadata & progress plan

* Fix CI snapshot and docs checks
2026-05-16 11:39:55 +02:00
Andrey Avtomonov
a72fca2b32
fix(cli): auto-install runtime during setup (#116)
* fix(cli): auto-install runtime during setup

* test: align docs smoke with readme
2026-05-16 11:39:43 +02:00
Luca Martial
703cbd92fc
refactor(cli): remove stale setup context detach state (#109) 2026-05-15 07:09:58 -04:00
Andrey Avtomonov
f8db99811a
feat(context): add driver-discriminated connection schemas (#96)
* refactor(context): export and describe mapping shape schemas

* feat(context): add driver-schemas module with warehouse drivers

* feat(context): add metabase, looker, lookml driver schemas with mappings

* feat(context): add notion, dbt, metricflow driver schemas

* refactor(context): make connectionSchema a driver-discriminated union

* chore(context): re-export KtxConnectionConfig from project package

* docs(context): add connection driver schema plan

* chore(secrets): allowlist example credentials in driver-schemas fixtures

* test(cli): align metabase fixtures with required api_url field

The driver-discriminated union added in this branch now requires api_url
for metabase connections and a known driver for warehouses. Update slow
CLI test fixtures and assertions so they exercise the new schema:
- ingest.test-utils.ts: add api_url to the prod-metabase fixture.
- setup.test.ts: switch metabase fixture from 'url' to 'api_url'.
- local-scan-connectors.test.ts: invalid-driver/missing-driver tests now
  expect the schema error from loadKtxProject (parse-time rejection).
2026-05-15 00:08:11 +02:00
Andrey Avtomonov
ce23aca4c4
fix: remove project from ktx config (#95) 2026-05-14 17:39:31 +02:00
Andrey Avtomonov
6c4623f2ff
feat(cli): enforce required database selection and improve tree-picker UX (#86)
* feat(cli): enforce required database selection and improve tree-picker UX

- Require at least one database driver via prompt `required: true` instead of
  looping on empty selection; remove the now-dead retry/back-on-empty branch.
- Surface the recommended option with a "(recommended)" hint in the depth and
  query-history prompts.
- Tree picker: add `◧` partial glyph for parents whose descendants are checked,
  and make `a` toggle select-all-visible / select-none.

* fix(cli): drop unused export from tree-picker toggleSelectAllVisible

Knip flagged the export as unused; the function is only consumed by the
internal reducer via the 'toggle-select-all-visible' command, so demote
it to a module-local helper to keep CI's dead-code check green.

* test(cli): drop empty-selection warning assertion from setup test

The empty-selection retry/warning loop in `chooseDrivers` was removed in
favor of `multiselect`'s `required: true`, so the legacy warning string
is unreachable. Update the test to assert the simpler back-from-selection
return-to-embeddings flow.
2026-05-14 14:35:58 +02:00
Andrey Avtomonov
b00c1a11a9
feat: merge ingest and scan
* docs: add CLI component reuse guidance

* docs: add unified ingest ux design

* Refine unified ingest UX design after adversarial review iteration 1

* Refine unified ingest UX design after adversarial review iteration 2

* Refine unified ingest UX design after adversarial review iteration 3

* feat(cli): route public connection ingest command

* feat(cli): hide standalone scan from public help

* feat(cli): plan public ingest depth and query history

* feat(cli): execute public database ingest facets

* feat(ingest): read connection query history config

* fix(cli): use public ingest wording

* fix(config): stop generating ingest adapter allow lists

* docs: document public ingest command

* test: align ingest surface expectations

* docs: add unified ingest public CLI surface plan

* feat(cli): preflight deep public ingest readiness

* feat(setup): store query history in connection context

* feat(setup): store database context depth

* feat(setup): verify context readiness by database depth

* fix(setup): keep context build foreground only

* fix(config): reject reserved ingest connection ids

* test: close unified ingest v1 expectations

* docs: add unified ingest v1 closure plan

* fix(ingest): bypass adapter allow-list for public source ingest

* fix(ingest): honor query history window intent

* fix(ingest): hide scan internals from public database ingest

* feat(ingest): use foreground view for interactive public ingest

* fix(setup): use schema context and query history wording

* test(cli): verify unified ingest public output

* docs: add unified ingest v1 public output closure plan

* fix(setup): forward query history flags

* fix(setup): prompt for postgres query history

* fix(status): report query history readiness

* fix(ingest): remove legacy public guidance

* fix(ingest): polish foreground retry copy

* docs(examples): use unified query history wording

* chore(ingest): finish public query history cleanup

* docs: add unified ingest v1 query history status cleanup plan

* test(docs): cover unified ingest public docs

* docs: align ingest CLI reference with unified UX

* docs: update context build guides for unified ingest

* docs: update setup and primary source ingest wording

* docs: stop advertising adapter-backed example ingest

* docs: close unified ingest public docs gaps

* docs: add unified ingest v1 docs site closure plan

* fix: render unified ingest foreground warnings

* fix: explain query history schema order

* fix: add public ingest retry guidance

* fix: align setup next steps with unified ingest

* fix: remove scan wording from demo progress

* test: verify unified ingest ux closure

* docs: add unified ingest v1 foreground and retry closure plan

* fix(cli): preserve query-history pull config in public ingest

* fix(cli): omit hidden commands from docs command tree

* test(cli): close unified ingest final public surface checks

* docs: add unified ingest v1 final public surface closure plan

* fix(cli): use public source labels in ingest reports

* fix(cli): suppress low-level public ingest output

* test(cli): verify unified ingest public plain output

* docs: add unified ingest v1 public plain output closure plan

* fix(cli): add public ingest copy sanitizers

* fix(cli): sanitize public ingest progress copy

* fix(cli): rename setup schema scope prompt

* docs(plan): add progress copy closure; test: align setup back-nav fixture

Adds the iter9 plan and updates the setup back-navigation test fixture
to pass disableQueryHistory plus listSchemas/listTables stubs that the
unified ingest setup step now requires.

* docs(plan): add final ux labels plan with narrowed label scans

* fix(cli): aggregate unsupported query-history warnings

* fix(cli): align setup database labels

* test(cli): fix setup database test type-check

* fix(cli): remove primary-source wording from setup output

* test(cli): verify unified ingest setup closure

* docs(plan): add unified ingest v1 verification copy closure plan

* fix(cli): remove top-level scan command

* fix(cli): remove legacy ingest and wiki commands

* Merge scan into ingest flow

* feat(cli): split ingest progress into per-phase rows, rename work units to tasks

Each database target in the unified ingest dashboard now renders one row per
real subprocess (Schema, then Query history when enabled) instead of a single
combined bar. Each phase has its own monotonic 0-100% bar so the progress
never snaps back to zero when historic-sql starts after scan completes.
Completed phases keep their final bar, summary, and elapsed time visible as
an inline audit trail; queued and skipped phases are shown explicitly.

Also rename user-facing "work units" / "Failed work units" to "tasks" /
"Failed tasks" in ingest output and parseIngestSummary. The parser still
accepts the legacy "Work units:" wording in captured output for backward
compat. Internal memory-flow event names and type fields are left alone.

* Fix test harness failures

* Fix CI smoke checks

---------

Co-authored-by: Andrey Avtomonov <7889985+andreybavt@users.noreply.github.com>
2026-05-14 01:43:06 +02:00
Andrey Avtomonov
3fde4438b1
fix: stop requiring readonly connection config (#71) 2026-05-13 19:37:25 +02:00
Luca Martial
4973ca562f
Restore Vertex AI LLM setup (#56)
* feat(context): resolve Vertex AI config references

* feat(cli): restore Vertex AI LLM setup

---------

Co-authored-by: Andrey Avtomonov <andreybavt@gmail.com>
2026-05-13 14:42:38 +02:00
Andrey Avtomonov
b75576279c
fix: store Metabase mappings in ktx.yaml (#61)
* fix: store Metabase mappings in ktx.yaml

* docs: note KTX has no public users

* refactor: drop setup progress compatibility
2026-05-13 13:55:21 +02:00
Andrey Avtomonov
b9e0a746af
feat(cli): clean up dev command surface (#57)
* feat(cli): clean up dev command surface

* test: align CI expectations with CLI cleanup

* test(cli): update slow test command expectations
2026-05-13 12:00:08 +02:00
Luca Martial
fcdf5234c6
Merge pull request #45 from Kaelio/luca/klo-654-improve-indents
feat(cli): add box-drawing prefixes to setup messages
2026-05-12 19:58:55 -04:00
Luca Martial
509f9f5301 feat(cli): prefix text-input continuation lines with box-drawing characters
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 16:58:00 -07:00
Luca Martial
e13350c970
Merge pull request #47 from Kaelio/luca-martial/save-setup-in-dot-ktx
Save setup completion state in .ktx/setup/state.json
2026-05-12 19:27:26 -04:00
Luca Martial
dbfee6b453 feat(cli): migrate all setup steps to use local state for completion tracking
Update every setup step to write completed_steps to .ktx/setup/state.json
instead of ktx.yaml, stripping legacy entries from config on write.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 16:26:23 -07:00
Andrey Avtomonov
17a2fee69a
fix(cli): remove ktx setup subcommands (#42)
* fix(cli): remove ktx setup subcommands

* test(scripts): update setup-dev status expectation
2026-05-13 00:38:26 +02:00
Andrey Avtomonov
e15a4ebaec feat(cli): clean up command surface 2026-05-12 23:51:46 +02:00
Luca Martial
60457e9407
Improve schema setup and Notion ingest UX (#14)
* Improve schema setup and Notion ingest UX

* Handle Postgres network scan failures

* WIP: save local changes before main merge

* Refine setup prompt choices

* Tighten ingest reconciliation guidance

* Commit setup config updates

* Canonicalize unmapped fallback details

* Count reconciliation actions in reports

* Harden semantic layer source validation

* Return wiki content after edits

* Validate SL sources against manifests

* Validate wiki refs before writes

* Simplify CLI next steps

* Clarify agent setup summary

* Surface dbt target SL sources

* Recover SL write fallbacks

* Preserve failed context build metadata

* Track raw paths for ingest actions

* test(cli): update seeded demo expectations

* fix(ingest): scope fallback recovery checks

* fix(sl): tighten source validation guards

* fix(wiki): ignore empty embedding vectors

* Improve Notion ingest UX

* Enforce flat wiki keys

* test(context): update wiki key assertion

---------

Co-authored-by: Andrey Avtomonov <andreybavt@gmail.com>
2026-05-12 22:56:58 +02:00
Andrey Avtomonov
60b29bb1e6 Merge origin/main into fix-metabase-readiness 2026-05-12 10:28:35 +02:00
Andrey Avtomonov
bb8868f238
Merge pull request #19 from Kaelio/andreybavt/dbt-vertex-no-anthropic
fix(cli): honor configured LLM backends in setup
2026-05-12 10:25:24 +02:00
Luca Martial
950fa151ce
Merge pull request #26 from Kaelio/luca-martial/demo-data-wizard-ux
feat(cli): add demo guided tour with real Orbit project data
2026-05-12 01:06:45 -04:00
Luca Martial
6943f1708d refactor(cli): remove dead demo dep from KtxSetupDeps
The demo entry menu now calls runDemoTour directly, so the injectable
demo property and KtxDemoArgs import are unused. Update test to mock the
new module import.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-11 22:03:20 -07:00
Luca Martial
6629d69ab4 refactor(cli): rename demo menu option to "Explore a pre-built KTX project"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-11 16:13:30 -07:00
Andrey Avtomonov
e05a6d43ab fix(cli): report metabase ingest readiness 2026-05-12 01:07:47 +02:00
Andrey Avtomonov
02e096a8e3 fix(cli): honor configured llm backends in setup 2026-05-12 01:05:28 +02:00
Andrey Avtomonov
1bd29c7eb1 Fix historic SQL ingest setup and progress 2026-05-11 22:35:07 +02:00
Andrey Avtomonov
9dad936ac7
feat: npm-managed Python runtime for @kaelio/ktx (#7)
* docs: add npm managed python runtime design

* build: add bundled python runtime wheel builder

* build: make local embedding dependencies optional

* build: bundle python runtime wheel in cli artifacts

* build: track bundled python runtime release artifact

* test: verify bundled python runtime wheel

* docs: add plan for bundled python runtime wheel

* test: cover managed python runtime lifecycle

* feat: add managed python runtime installer

* feat: add runtime command runner

* feat: expose runtime management commands

* test: verify managed python runtime commands

* docs: add plan for managed python runtime installer

* feat: add managed python command helper

* feat: use managed runtime for sl query compute

* feat: route sl query managed runtime policy

* docs: add plan for managed runtime sl query integration

* feat: add managed runtime daemon metadata

* feat: manage python daemon lifecycle

* feat: add runtime daemon start stop commands

* fix: verify managed runtime daemon lifecycle

* docs: add plan for managed runtime daemon lifecycle

* feat: add managed local embeddings config marker

* feat: add managed local embeddings daemon helper

* feat: use managed runtime for local embedding setup

* feat: pass managed runtime policy through setup

* docs: add plan for managed local embeddings runtime

* feat: read CLI package metadata dynamically

* feat: assemble public kaelio ktx npm package

* feat: release one public kaelio ktx npm artifact

* test: cover public kaelio ktx package invocations

* chore: verify public kaelio ktx package artifacts

* docs: add plan for public kaelio ktx npm package

* test: verify managed runtime in public package smoke

* test: finalize managed runtime release smoke

* docs: add plan for managed runtime release smoke

* test: specify local embeddings release smoke

* feat: add local embeddings runtime smoke

* chore: register local embeddings smoke

* fix: verify local embeddings smoke

* fix: restore artifact smoke python env helper

* docs: add plan for managed local embeddings release smoke

* refactor: share managed runtime install policy parsing

* feat: use managed runtime for agent semantic queries

* feat: use managed runtime for MCP semantic compute

* docs: add plan for managed agent and MCP semantic runtime

* feat(cli): add managed daemon HTTP helpers

* feat(cli): route local adapters through managed daemon

* feat(cli): use managed daemon for ingest helpers

* feat(cli): pass managed daemon options to scan

* feat(context): pass MCP ingest pull config options

* feat(cli): pass managed daemon options to serve ingest

* test: verify managed local ingest daemon runtime

* docs: add plan for managed local ingest daemon runtime

* docs: align managed runtime examples

* docs: add plan for managed runtime docs cleanup

* test: cover published package runtime smoke commands

* test: validate published package smoke outputs

* docs: add plan for published package runtime smoke

* build: stamp public npm package version

* release: add npm public release policy

* release: add guarded npm publish script

* release: document public npm release handoff

* docs: add plan for public npm release handoff

* test: cover managed runtime prune in package smoke

* docs: document managed runtime prune

* docs: add plan for managed runtime prune smoke and docs

* chore: encode uv runtime prerequisite policy

* fix: clarify missing uv runtime error

* docs: document uv runtime prerequisite

* docs: add plan for uv runtime prerequisite contract

* refactor: limit release artifacts to public package runtime

* chore: align release policy with bundled runtime wheel

* docs: describe single public runtime artifact surface

* test: verify single public runtime artifact contract

* docs: add plan for single public runtime artifact cleanup

* fix: align local embeddings smoke with public version

* docs: add plan for local embeddings smoke public version

* release: soft-launch as @kaelio/ktx@0.1.0-rc.0 on next tag

Publish target moves to the pre-release version 0.1.0-rc.0 under the next
dist-tag so npm install @kaelio/ktx (which resolves to latest) does not
pick up the soft-launch build. Users opt in via @kaelio/ktx@next.

* Fix release script boundary checks

* Remove PostHog from public package bundle
2026-05-11 15:50:34 +02:00
Luca Martial
c82989119b Update setup and ingest flows 2026-05-10 23:13:17 -07:00
Luca Martial
549fb35e75 Show progress when watching context builds 2026-05-10 17:08:55 -07:00
Andrey Avtomonov
3ce510b55b rename klo to ktx 2026-05-10 23:51:24 +02:00
Andrey Avtomonov
1a42152e6f Initial open-source release 2026-05-10 23:12:26 +02:00