Commit graph

319 commits

Author SHA1 Message Date
Andrey Avtomonov
644659fc1b
Merge branch 'main' into explore-research-agent-tools 2026-05-15 00:08:24 +02:00
Andrey Avtomonov
f8db99811a
feat(context): add driver-discriminated connection schemas (#96)
* refactor(context): export and describe mapping shape schemas

* feat(context): add driver-schemas module with warehouse drivers

* feat(context): add metabase, looker, lookml driver schemas with mappings

* feat(context): add notion, dbt, metricflow driver schemas

* refactor(context): make connectionSchema a driver-discriminated union

* chore(context): re-export KtxConnectionConfig from project package

* docs(context): add connection driver schema plan

* chore(secrets): allowlist example credentials in driver-schemas fixtures

* test(cli): align metabase fixtures with required api_url field

The driver-discriminated union added in this branch now requires api_url
for metabase connections and a known driver for warehouses. Update slow
CLI test fixtures and assertions so they exercise the new schema:
- ingest.test-utils.ts: add api_url to the prod-metabase fixture.
- setup.test.ts: switch metabase fixture from 'url' to 'api_url'.
- local-scan-connectors.test.ts: invalid-driver/missing-driver tests now
  expect the schema error from loadKtxProject (parse-time rejection).
2026-05-15 00:08:11 +02:00
Luca Martial
d244261aa7
docs: add context layer diagram (#102) 2026-05-14 17:57:16 -04:00
Andrey Avtomonov
d431fbfa5d
feat(docs-site): refresh nav mascot with SVG and bump size (#101)
Replace the PNG mascot with the refined "C" SVG (light + dark variants)
and enlarge the nav logo from 32px to 56px so it reads at a glance.
Also drop the same SVG pair into assets/ for repo-wide reuse.
2026-05-14 23:45:41 +02:00
Andrey Avtomonov
13ebe4fb6f chore: build runtime artifacts in conductor setup 2026-05-14 23:08:33 +02:00
Andrey Avtomonov
6c73029d0c Merge remote-tracking branch 'origin/main' into explore-research-agent-tools
# Conflicts:
#	packages/cli/src/print-command-tree.test.ts
#	packages/context/skills/sl_capture/SKILL.md
2026-05-14 22:05:00 +02:00
Andrey Avtomonov
8fd8ae6388 docs: add research-agent MCP ingest contract convergence plan 2026-05-14 19:27:38 +02:00
Andrey Avtomonov
fd3a25f14e docs(context): update ingest verification prompts for connectionId 2026-05-14 19:25:59 +02:00
Andrey Avtomonov
e4f2863fed refactor(context): use connectionId in warehouse verification tools 2026-05-14 19:25:03 +02:00
Andrey Avtomonov
6179667e45 docs: add research-agent MCP setup-agents plan 2026-05-14 19:11:30 +02:00
Andrey Avtomonov
c35aa760e6 feat(cli): support Claude local MCP setup scope 2026-05-14 19:09:48 +02:00
Andrey Avtomonov
6cb03d6924 feat(cli): configure MCP clients in setup agents 2026-05-14 19:07:22 +02:00
Andrey Avtomonov
0955b36887 feat(cli): install KTX research skill 2026-05-14 19:05:46 +02:00
Andrey Avtomonov
d79c51abaa docs: add research-agent MCP http daemon plan 2026-05-14 18:55:22 +02:00
Andrey Avtomonov
88f91f27c2 fix(cli): stabilize mcp daemon verification 2026-05-14 18:54:18 +02:00
Luca Martial
5cf2c89093
Revise CLI reference docs (#100)
* docs: revise CLI reference

* docs: sync CLI reference with current commands
2026-05-14 12:53:55 -04:00
Andrey Avtomonov
a9c7c152f1
docs(agents): instruct agents to update docs-site after code changes (#99) 2026-05-14 18:53:44 +02:00
Andrey Avtomonov
3bba9eec79 feat(cli): add ktx mcp commands 2026-05-14 18:52:00 +02:00
Andrey Avtomonov
db09df4d72 feat(cli): manage mcp daemon lifecycle 2026-05-14 18:50:54 +02:00
Andrey Avtomonov
6bff3c3492 feat(cli): host mcp over streamable http 2026-05-14 18:50:08 +02:00
Andrey Avtomonov
7ffa99983f feat(cli): add mcp http security helpers 2026-05-14 18:47:50 +02:00
Luca Martial
372c90b533
Polish documentation copy (#98) 2026-05-14 12:43:14 -04:00
Andrey Avtomonov
e974f3e59f docs: add research-agent MCP discover_data plan 2026-05-14 18:38:18 +02:00
Andrey Avtomonov
fc903f32b7 feat: wire local discover data MCP port 2026-05-14 18:36:18 +02:00
Andrey Avtomonov
e74976d321 feat: expose discover data MCP tool 2026-05-14 18:35:30 +02:00
Andrey Avtomonov
6a61f209e2 feat: add MCP discover data service 2026-05-14 18:34:44 +02:00
Andrey Avtomonov
f1c073b614 docs: add research-agent MCP dictionary_search plan 2026-05-14 18:24:48 +02:00
Andrey Avtomonov
b8418c7a79 feat(context): expose local MCP dictionary search 2026-05-14 18:22:55 +02:00
Andrey Avtomonov
edb62deed2 feat(context): register MCP dictionary search tool 2026-05-14 18:22:25 +02:00
Andrey Avtomonov
d0b8996456 feat(context): add dictionary search service 2026-05-14 18:21:52 +02:00
Andrey Avtomonov
ff3a8c5777 docs: add research-agent MCP entity_details plan 2026-05-14 18:14:17 +02:00
Andrey Avtomonov
a27400b3dd test(context): align entity details scan fixtures 2026-05-14 18:13:23 +02:00
Andrey Avtomonov
da6f8873d4 feat(context): expose local MCP entity details 2026-05-14 18:11:27 +02:00
Andrey Avtomonov
9d9fa9bc3b feat(context): register MCP entity details tool 2026-05-14 18:10:03 +02:00
Andrey Avtomonov
700c0ba5d7 feat(context): add scan-backed entity details service 2026-05-14 18:09:16 +02:00
Andrey Avtomonov
371fcd47eb docs: add research-agent MCP sql execution foundation plan 2026-05-14 18:00:40 +02:00
Andrey Avtomonov
66e7c9fca4 test(context): update SQL analysis port fixtures 2026-05-14 17:59:21 +02:00
Andrey Avtomonov
807f86d761 feat(context): execute MCP SQL through validated connector path 2026-05-14 17:57:41 +02:00
Andrey Avtomonov
c774870346 feat(context): register MCP sql execution tool 2026-05-14 17:56:33 +02:00
Andrey Avtomonov
06f020dca1 feat(context): expose read-only SQL validation port 2026-05-14 17:55:23 +02:00
Andrey Avtomonov
aa4431b295 feat(daemon): validate read-only SQL with sqlglot 2026-05-14 17:54:36 +02:00
Andrey Avtomonov
ce23aca4c4
fix: remove project from ktx config (#95) 2026-05-14 17:39:31 +02:00
Andrey Avtomonov
de9f4d97e7 Refine spec: drop connectionName compat carve-out and ground summary/snippet provenance per kind 2026-05-14 17:24:23 +02:00
Andrey Avtomonov
bef9d14b90 Refine research-agent MCP tools spec after adversarial review iteration 3 2026-05-14 17:06:35 +02:00
Andrey Avtomonov
1a79c0efc3 Refine research-agent MCP tools spec after adversarial review iteration 2 2026-05-14 16:58:28 +02:00
Andrey Avtomonov
9e6511182e Refine research-agent MCP tools spec after adversarial review iteration 1 2026-05-14 16:47:57 +02:00
Andrey Avtomonov
97ac2139bc docs(specs): design research-agent MCP tools and ktx mcp daemon
Adds the 2026-05-14 design spec for exposing four new MCP tools
(discover_data, entity_details, dictionary_search, sql_execution),
shipping a ktx-research skill, and introducing an HTTP-only ktx mcp
daemon so external agents can use KTX as a research-capable context
layer.
2026-05-14 16:42:40 +02:00
Andrey Avtomonov
2bca308863
feat(cli): add ktx dev schema to emit ktx.yaml JSON Schema (#93)
Annotates the Zod config schema with .describe() text on every field and
adds generateKtxProjectConfigJsonSchema() plus a ktx dev schema command
that prints (or writes) a draft-07 JSON Schema for editors and LLM agents.
2026-05-14 16:21:29 +02:00
Andrey Avtomonov
c7c5f63a66
feat(cli): extend ktx connection test to every supported driver (#92)
* feat(cli): extend `ktx connection test` to every supported driver

Dispatch by driver: native DBs now call `connector.testConnection()`
(was `introspect(dryRun)`), looker/notion/metabase hit their auth
endpoints, and dbt/metricflow/lookml run `git ls-remote` via the
existing `testRepoConnection` helper. Unknown drivers exit 1 with a
listing of supported ones.

* feat(cli): add `ktx connection test --all` summary list

Tests every configured connection in parallel and renders a single
Clack-style list (◇/│/◆/└, green ✓ / red ✗) consistent with sl list,
with per-row detail and a passed/failed footer. Exits non-zero if any
connection fails. Single-id `ktx connection test` output is preserved.

* fix(cli): read metabase status url from api_url

`ktx status` was probing `url` / `base_url` on metabase connections, but
ktx.yaml stores it as `api_url`, so the field always reported "url not
set". Read `api_url` directly and align the warning text with the actual
key.
2026-05-14 16:21:18 +02:00
Andrey Avtomonov
b3be54e3fa
refactor(context): validate ktx.yaml with Zod and surface issues in status (#91)
* refactor(context): validate ktx.yaml with Zod and surface issues in status

- Replace hand-rolled ktx.yaml parsing with a strict Zod schema and
  derive KtxProjectConfig types from it.
- Add validateKtxProjectConfig returning structured KtxConfigIssue[]
  with migration hints for deprecated keys (ingest.llm,
  scan.enrichment.backend, etc.).
- Wire ktx status/doctor to run validation, render schema issues in
  plain and JSON output, and add a Config row to project status.
- Update the orbit example to camelCase scan.relationships keys to
  match the schema.

* fix(context): tolerate legacy setup.completed_steps and optional driver

- Accept and drop the legacy setup.completed_steps field so existing
  ktx.yaml files migrated from older versions still load.
- Make connections.<id>.driver optional in the schema; runtime code
  already produces a clear "no driver" error at use time.

* feat(cli): add ktx status --validate to run only ktx.yaml schema validation

- New --validate flag dispatches a focused runKtxDoctor 'validate' branch
  that reads ktx.yaml, runs validateKtxProjectConfig, and skips LLM,
  connection, embedding, and query-history checks.
- Plain output prints a single Config row; JSON output emits
  {ok: true} on success or the existing invalid_config / missing_project
  shapes on failure.
2026-05-14 15:36:35 +02:00