* docs: align docs with current KTX behavior
* fix: generate valid agent sl query command
* docs: clarify KTX product mechanics
* fix: use <ol> for runtime pipeline steps in product mechanics
The PipelineStep component renders <li> elements, but the RuntimeDiagram
wrapper was a plain <div> instead of a list element. This produced invalid
HTML and accessibility warnings. IngestionDiagram already used <ol>.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add docs favicon
* docs: add semantic layer internals concept
* docs: refine documentation source label
* docs: clarify company documentation examples
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The staleness check compared source mtimes against packages/cli/dist/bin.js,
but tsc only rewrites outputs whose source actually changed. Editing any
non-bin source (e.g. setup.ts) left bin.js untouched, so its mtime stayed
older than the sources forever and every `pnpm run ktx` invocation
rebuilt the whole workspace. Write a dedicated .ktx-build-stamp after a
successful build and check sources against that instead.
* refactor(cli): unify output formatting across search and status commands
Replace clack-style box borders (◇/│/└) and bullets (●/◆) in printList
pretty mode with a clean status-style layout: bold headers, indented
aligned rows, no decorative framing. Migrate status-project.ts from
hand-rolled ANSI escape codes to shared symbols.ts color helpers.
Remove dead clack symbols from SYMBOLS, add yellow() for warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(cli): update stale badge role docstring after dim removal
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Stack a small "by Kaelio" line under the "KTX" wordmark in the docs site
nav logo, scale the mascot + wordmark ~1.4x, and fix the mascot asset
paths to include the /ktx basePath so they load in production. Also
ignore .playwright-cli/ session artifacts produced by local UI checks.
Drop the duplicate `pnpm run build` (artifacts:build already builds every
package). Run package builds in parallel topology via one recursive pnpm
invocation. Enable incremental tsc and keep the cli's tsbuildinfo outside
its dist (moved the dist wipe into a separate `clean` script). Run the
final `ktx status` doctor from a temp dir so it stops walking up into a
parent ktx.yaml and failing the script.
Conductor setup drops from ~26s to ~9.8s cold and ~4.4s warm.
- Add Next.js basePath '/ktx' so the docs site mounts under /ktx,
enabling it to be reverse-proxied by docs.kaelio.com.
- Add host-based redirects so docs.ktx.sh and ktx.sh permanently
redirect to https://docs.kaelio.com/ktx, making docs.kaelio.com
the single canonical home for KTX docs.
- Update markdown-preference middleware to be basePath-aware so the
llms.mdx rewrite still fires on /ktx/docs/* requests.
* fix(context): merge overlay columns onto manifest columns by name
composeOverlay was appending overlay columns to the manifest column list,
producing duplicate entries when dbt/metabase overlays declared a column
just to attach descriptions. The duplicates carried no `type`, so the
pydantic SourceDefinition rejected them at semantic-query time and broke
`ktx sl query` for every overlay-backed measure. Now overlay columns
match base columns by name (case-insensitive): same-name entries merge
onto the manifest (overlay fields win, type/role fall back to the base,
descriptions merge per source key) and only new names append.
* refactor(sl): split overlay columns from column_overrides and enforce TS/Python wire contract
Overlay sources now have two distinct collections: `columns:` for computed
columns (requiring `expr` + `type`) and `column_overrides:` for metadata
patches to inherited manifest columns. Composing or loading an overlay that
mixes the two — or references an unknown column — fails with a typed error.
Introduce `ResolvedSemanticLayerSource` / `resolvedSourceSchema` /
`toResolvedWire` as the strict shape sent to the Python engine, and add a
schema contract test that diffs Zod against the Pydantic JSON schema dumped
by `python -m semantic_layer dump-schema`. `SourceDefinition` is now
`extra="forbid"` on the Python side.
`loadAllSources` surfaces per-file load errors instead of swallowing them,
so validation/query paths can report manifest shard parse failures.
* fix(context): make scan description generation resilient and quiet
A transient sampleTable failure during ingest used to take out every
table in a connection: generateTableDescription returned a hardcoded
'Table not found' string into descriptions.ai, and KtxDescriptionGenerator
was constructed without a logger, so the failure left no trail anywhere.
- sampleTable / sampleColumn calls retry 3x with 200/400/800ms backoff,
honouring KtxScanContext.signal via a new KtxAbortedError.
- On retry exhaustion or missing capability, table generation falls back
to a metadata-only prompt built from column name / native type / comment
/ rawDescriptions. The column path follows the same rule -- call the
LLM when any of samples or rawDescriptions are available; skip only
when both are absent.
- Logger is now threaded from KtxScanContext into the generator. Failures
emit structured KtxScanWarning entries (new description_fallback_used
code, plus existing sampling_failed / enrichment_failed /
connector_capability_missing). ktx scan groups warnings by code so a
batch of identical failures collapses to one summary line plus sample.
- Returns null on failure instead of the 'Table not found' sentinel; the
manifest writer's existing guard already skips empty descriptions, so
schema YAML no longer carries misleading text. SCAN_MANAGED_DESCRIPTION_KEYS
already strips stale 'ai' on merge, so existing YAML clears on next run.
Also suppress AI SDK v6 'system in messages' warning: pull system messages
out of KtxMessageBuilder.wrapSimple's output via a new splitKtxSystemMessages
helper and pass them top-level to generateText (preserves cacheControl
providerOptions on the SystemModelMessage). Agent-runner's local
splitSystemPromptMessages dedupes onto the shared helper.
* test(docs): align examples-docs assertions with revamped docs
PR #103 (setup/guide doc revamp) reworded several CLI examples and
connection labels; the assertions in scripts/examples-docs.test.mjs
still referenced the pre-revamp wording and were failing in CI on main.
Update the regexes to match the post-revamp content:
- drop the `--json` flag from the sl-query example expectation
- move the `Driver:` / `Status: ok` probe to the connection reference,
which is where that output now lives (driver id is lowercase
`postgres`, not the display name `PostgreSQL`)
- drop the obsolete `Install \`uv\`...` troubleshooting line
- accept `<connectionId>` everywhere; the docs no longer use the
hyphenated `<connection-id>` form
- match the `warehouse` connection id used in the quickstart instead of
the `postgres-warehouse` id only used in the README and setup ref
* fix(sl): skip TS/Python schema contract test when uv is unavailable
The TypeScript checks CI job does not install uv or Python, so the
module-level `execFileSync('uv', ...)` in schemas.contract.test.ts threw
ENOENT and failed the suite. Wrap the schema dump in a try/catch and
guard the describe block with `describe.skipIf` so the test skips in
environments without uv. Local dev and any CI job that has uv on PATH
still runs the cross-language contract assertion.
* refactor(context): export and describe mapping shape schemas
* feat(context): add driver-schemas module with warehouse drivers
* feat(context): add metabase, looker, lookml driver schemas with mappings
* feat(context): add notion, dbt, metricflow driver schemas
* refactor(context): make connectionSchema a driver-discriminated union
* chore(context): re-export KtxConnectionConfig from project package
* docs(context): add connection driver schema plan
* chore(secrets): allowlist example credentials in driver-schemas fixtures
* test(cli): align metabase fixtures with required api_url field
The driver-discriminated union added in this branch now requires api_url
for metabase connections and a known driver for warehouses. Update slow
CLI test fixtures and assertions so they exercise the new schema:
- ingest.test-utils.ts: add api_url to the prod-metabase fixture.
- setup.test.ts: switch metabase fixture from 'url' to 'api_url'.
- local-scan-connectors.test.ts: invalid-driver/missing-driver tests now
expect the schema error from loadKtxProject (parse-time rejection).
