Stack a small "by Kaelio" line under the "KTX" wordmark in the docs site
nav logo, scale the mascot + wordmark ~1.4x, and fix the mascot asset
paths to include the /ktx basePath so they load in production. Also
ignore .playwright-cli/ session artifacts produced by local UI checks.
Drop the duplicate `pnpm run build` (artifacts:build already builds every
package). Run package builds in parallel topology via one recursive pnpm
invocation. Enable incremental tsc and keep the cli's tsbuildinfo outside
its dist (moved the dist wipe into a separate `clean` script). Run the
final `ktx status` doctor from a temp dir so it stops walking up into a
parent ktx.yaml and failing the script.
Conductor setup drops from ~26s to ~9.8s cold and ~4.4s warm.
- Add Next.js basePath '/ktx' so the docs site mounts under /ktx,
enabling it to be reverse-proxied by docs.kaelio.com.
- Add host-based redirects so docs.ktx.sh and ktx.sh permanently
redirect to https://docs.kaelio.com/ktx, making docs.kaelio.com
the single canonical home for KTX docs.
- Update markdown-preference middleware to be basePath-aware so the
llms.mdx rewrite still fires on /ktx/docs/* requests.
* fix(context): merge overlay columns onto manifest columns by name
composeOverlay was appending overlay columns to the manifest column list,
producing duplicate entries when dbt/metabase overlays declared a column
just to attach descriptions. The duplicates carried no `type`, so the
pydantic SourceDefinition rejected them at semantic-query time and broke
`ktx sl query` for every overlay-backed measure. Now overlay columns
match base columns by name (case-insensitive): same-name entries merge
onto the manifest (overlay fields win, type/role fall back to the base,
descriptions merge per source key) and only new names append.
* refactor(sl): split overlay columns from column_overrides and enforce TS/Python wire contract
Overlay sources now have two distinct collections: `columns:` for computed
columns (requiring `expr` + `type`) and `column_overrides:` for metadata
patches to inherited manifest columns. Composing or loading an overlay that
mixes the two — or references an unknown column — fails with a typed error.
Introduce `ResolvedSemanticLayerSource` / `resolvedSourceSchema` /
`toResolvedWire` as the strict shape sent to the Python engine, and add a
schema contract test that diffs Zod against the Pydantic JSON schema dumped
by `python -m semantic_layer dump-schema`. `SourceDefinition` is now
`extra="forbid"` on the Python side.
`loadAllSources` surfaces per-file load errors instead of swallowing them,
so validation/query paths can report manifest shard parse failures.
* fix(context): make scan description generation resilient and quiet
A transient sampleTable failure during ingest used to take out every
table in a connection: generateTableDescription returned a hardcoded
'Table not found' string into descriptions.ai, and KtxDescriptionGenerator
was constructed without a logger, so the failure left no trail anywhere.
- sampleTable / sampleColumn calls retry 3x with 200/400/800ms backoff,
honouring KtxScanContext.signal via a new KtxAbortedError.
- On retry exhaustion or missing capability, table generation falls back
to a metadata-only prompt built from column name / native type / comment
/ rawDescriptions. The column path follows the same rule -- call the
LLM when any of samples or rawDescriptions are available; skip only
when both are absent.
- Logger is now threaded from KtxScanContext into the generator. Failures
emit structured KtxScanWarning entries (new description_fallback_used
code, plus existing sampling_failed / enrichment_failed /
connector_capability_missing). ktx scan groups warnings by code so a
batch of identical failures collapses to one summary line plus sample.
- Returns null on failure instead of the 'Table not found' sentinel; the
manifest writer's existing guard already skips empty descriptions, so
schema YAML no longer carries misleading text. SCAN_MANAGED_DESCRIPTION_KEYS
already strips stale 'ai' on merge, so existing YAML clears on next run.
Also suppress AI SDK v6 'system in messages' warning: pull system messages
out of KtxMessageBuilder.wrapSimple's output via a new splitKtxSystemMessages
helper and pass them top-level to generateText (preserves cacheControl
providerOptions on the SystemModelMessage). Agent-runner's local
splitSystemPromptMessages dedupes onto the shared helper.
* test(docs): align examples-docs assertions with revamped docs
PR #103 (setup/guide doc revamp) reworded several CLI examples and
connection labels; the assertions in scripts/examples-docs.test.mjs
still referenced the pre-revamp wording and were failing in CI on main.
Update the regexes to match the post-revamp content:
- drop the `--json` flag from the sl-query example expectation
- move the `Driver:` / `Status: ok` probe to the connection reference,
which is where that output now lives (driver id is lowercase
`postgres`, not the display name `PostgreSQL`)
- drop the obsolete `Install \`uv\`...` troubleshooting line
- accept `<connectionId>` everywhere; the docs no longer use the
hyphenated `<connection-id>` form
- match the `warehouse` connection id used in the quickstart instead of
the `postgres-warehouse` id only used in the README and setup ref
* fix(sl): skip TS/Python schema contract test when uv is unavailable
The TypeScript checks CI job does not install uv or Python, so the
module-level `execFileSync('uv', ...)` in schemas.contract.test.ts threw
ENOENT and failed the suite. Wrap the schema dump in a try/catch and
guard the describe block with `describe.skipIf` so the test skips in
environments without uv. Local dev and any CI job that has uv on PATH
still runs the cross-language contract assertion.
* refactor(context): export and describe mapping shape schemas
* feat(context): add driver-schemas module with warehouse drivers
* feat(context): add metabase, looker, lookml driver schemas with mappings
* feat(context): add notion, dbt, metricflow driver schemas
* refactor(context): make connectionSchema a driver-discriminated union
* chore(context): re-export KtxConnectionConfig from project package
* docs(context): add connection driver schema plan
* chore(secrets): allowlist example credentials in driver-schemas fixtures
* test(cli): align metabase fixtures with required api_url field
The driver-discriminated union added in this branch now requires api_url
for metabase connections and a known driver for warehouses. Update slow
CLI test fixtures and assertions so they exercise the new schema:
- ingest.test-utils.ts: add api_url to the prod-metabase fixture.
- setup.test.ts: switch metabase fixture from 'url' to 'api_url'.
- local-scan-connectors.test.ts: invalid-driver/missing-driver tests now
expect the schema error from loadKtxProject (parse-time rejection).
Replace the PNG mascot with the refined "C" SVG (light + dark variants)
and enlarge the nav logo from 32px to 56px so it reads at a glance.
Also drop the same SVG pair into assets/ for repo-wide reuse.
Annotates the Zod config schema with .describe() text on every field and
adds generateKtxProjectConfigJsonSchema() plus a ktx dev schema command
that prints (or writes) a draft-07 JSON Schema for editors and LLM agents.
* feat(cli): extend `ktx connection test` to every supported driver
Dispatch by driver: native DBs now call `connector.testConnection()`
(was `introspect(dryRun)`), looker/notion/metabase hit their auth
endpoints, and dbt/metricflow/lookml run `git ls-remote` via the
existing `testRepoConnection` helper. Unknown drivers exit 1 with a
listing of supported ones.
* feat(cli): add `ktx connection test --all` summary list
Tests every configured connection in parallel and renders a single
Clack-style list (◇/│/◆/└, green ✓ / red ✗) consistent with sl list,
with per-row detail and a passed/failed footer. Exits non-zero if any
connection fails. Single-id `ktx connection test` output is preserved.
* fix(cli): read metabase status url from api_url
`ktx status` was probing `url` / `base_url` on metabase connections, but
ktx.yaml stores it as `api_url`, so the field always reported "url not
set". Read `api_url` directly and align the warning text with the actual
key.
* refactor(context): validate ktx.yaml with Zod and surface issues in status
- Replace hand-rolled ktx.yaml parsing with a strict Zod schema and
derive KtxProjectConfig types from it.
- Add validateKtxProjectConfig returning structured KtxConfigIssue[]
with migration hints for deprecated keys (ingest.llm,
scan.enrichment.backend, etc.).
- Wire ktx status/doctor to run validation, render schema issues in
plain and JSON output, and add a Config row to project status.
- Update the orbit example to camelCase scan.relationships keys to
match the schema.
* fix(context): tolerate legacy setup.completed_steps and optional driver
- Accept and drop the legacy setup.completed_steps field so existing
ktx.yaml files migrated from older versions still load.
- Make connections.<id>.driver optional in the schema; runtime code
already produces a clear "no driver" error at use time.
* feat(cli): add ktx status --validate to run only ktx.yaml schema validation
- New --validate flag dispatches a focused runKtxDoctor 'validate' branch
that reads ktx.yaml, runs validateKtxProjectConfig, and skips LLM,
connection, embedding, and query-history checks.
- Plain output prints a single Config row; JSON output emits
{ok: true} on success or the existing invalid_config / missing_project
shapes on failure.
* fix(llm): wire prompt caching through all Anthropic call sites
- page-triage classifier + light-extraction now put the static skill
prompt in `system:` so the per-document caches hit instead of
re-sending boilerplate in the user message every call.
- Description generation builders return `{ system, user }` with
instruction text + word limit moved into the cacheable system.
- Relationship-LLM proposal framing moved to `system:`.
- `KtxMessageBuilder.wrapSimple` skips the history breakpoint for
single-message calls (cache write that could never be reused).
- Gateway backend now sets `anthropic-beta: extended-cache-ttl-2025-04-11`
so 1h TTLs don't silently downgrade to 5m on Gateway routes.
* fix(llm): keep wrapSimple history breakpoint so multi-step agent loops cache
Reverts the wrapSimple `messages.length > 1` guard from the prior commit.
agent-runner uses wrapSimple with a single user message, but generateText
runs a multi-step tool loop inside it — the cache marker on the first user
message is reused by every subsequent step, so it isn't waste.
The release validator (scripts/validate-llm-debug-jsonl.mjs) also requires
a `message-part` marker target in captured debug JSONL.
Routes `ktx wiki list` and `ktx wiki search` through the shared printList()
renderer so all four list/search commands now produce the same Clack-style
pretty box, TSV plain output, and JSON envelope. Adds a `--output` flag to
the wiki commands mirroring sl, and surfaces relevance score as a leading
dim badge ("87%") in pretty mode and a `score=` prefix in plain mode for
both wiki search and sl search. Empty results now emit a consistent
actionable hint across commands.
Wraps the validation clone in defaultValidateDbt so auth or network
failures surface as a clean validation error instead of an unhandled
RepoFetchError that exits the wizard. Verifies pasted tokens with
testGitRepo before saving them as a secret so bad tokens are caught at
paste time. In interactive setup, validation failures now bounce the
user back to source selection (with a "Edit the connection or pick a
different source" hint) instead of killing the process; --source flag
mode still exits with failed as before.
* feat(cli): enforce required database selection and improve tree-picker UX
- Require at least one database driver via prompt `required: true` instead of
looping on empty selection; remove the now-dead retry/back-on-empty branch.
- Surface the recommended option with a "(recommended)" hint in the depth and
query-history prompts.
- Tree picker: add `◧` partial glyph for parents whose descendants are checked,
and make `a` toggle select-all-visible / select-none.
* fix(cli): drop unused export from tree-picker toggleSelectAllVisible
Knip flagged the export as unused; the function is only consumed by the
internal reducer via the 'toggle-select-all-visible' command, so demote
it to a module-local helper to keep CI's dead-code check green.
* test(cli): drop empty-selection warning assertion from setup test
The empty-selection retry/warning loop in `chooseDrivers` was removed in
favor of `multiselect`'s `required: true`, so the legacy warning string
is unreachable. Update the test to assert the simpler back-from-selection
return-to-embeddings flow.
