The MCP sql_execution/sl_query tools and the `ktx sql` CLI threw a plain Error naming no valid connection ids when an agent passed an unconfigured connectionId (or omitted it with multiple connections). The message reached the agent verbatim but gave it nothing to correct with, so it re-guessed for days, and each correct caller-driven rejection filed in PostHog Error Tracking as a ktx fault (issue 019eb10c, 8 occurrences on one install).
Add a shared resolver (resolveConfiguredConnection / resolveRequiredConnectionId) that throws KtxExpectedError listing the configured connections, and route the three SQL-execution call sites through it. Expected-error classification keeps these out of Error Tracking while the actionable message lets agents self-correct.
* fix: read semantic sources safely
* test: retarget reindex per-scope error case to a broken manifest
Reading a broken standalone source was made non-fatal in de1f1a8d (it is
surfaced for repair instead of throwing), so the reindex per-scope error
test no longer captured an error. Point it at a corrupt manifest shard,
which is the remaining fatal read failure the per-scope catch must
isolate, and assert the captured error names the offending file.
* fix(sl): decouple semantic-layer file names from warehouse naming rules
The in-file `name:` field is now the sole source identity; the filename is
a derived label that never participates in identity. This removes the
"Unsafe semantic-layer source name" failure class entirely: any warehouse
identifier (Snowflake's uppercase SIGNED_UP, EVENT$LOG, dotted names) can
be read, overlaid, edited, and deleted.
- New `source-files.ts`: one total filename derivation (safe lowercase
names verbatim; otherwise slug + sha256-hash suffix, immune to
case-insensitive-filesystem collisions) and one by-name file resolver.
- Reads resolve by name everywhere; the path-from-name fast path and
`assertSafeSourceName` are gone.
- Writes resolve-then-write: rewrites land on the file that declares the
name (human renames survive); new sources get a derived filename; a
derived path occupied by a different source fails instead of clobbering.
- `readSourceFile` returns null for missing files instead of forcing every
caller to launder IO errors; `deleteSource` distinguishes manifest-backed
sources from not-found instead of silently succeeding.
- `sl_write_source` accepts verbatim warehouse identifiers (snake_case is
now a recommendation for new sources) and rejects sourceName/source.name
mismatches; `sl_edit_source` rejects name-changing edits.
- Ingest projection commits, gate-repair allowlists, and touched-source
derivation use resolved paths / in-file names instead of interpolating
`<connId>/<name>.yaml`.
- Collapsed the five parallel path derivations and duplicated path-token
helpers onto the shared module; dropped dead service methods.
* fix(sl): resolve sources by declared name end-to-end and gate warehouse SQL with the parser-backed validator
- Key broken/renamed semantic-layer files by their recoverable in-file
name (slSourceNameForFile) so mid-edit sources stay reachable under
their real identity in reads, listings, and search
- Derive finalization touched sources from composed-source diffs and
recover deleted files' declared names from the pre-change commit
instead of parsing hash-derived filenames
- Resolve revert/rollback paths against history (listFilesAtCommit) so
human-renamed files are restored where they lived at preHead
- Validate ingest sql_execution through the daemon's sqlglot
validateReadOnly in the connection's dialect, sharing one
driver-to-dialect map (sql-analysis/dialect.ts) across MCP and ingest
- Harden the local read-only SQL backstop: accept leading comments,
reject smuggled second statements, and strip trailing
semicolons/comments before row-limit wrapping
* feat(setup): write per-role llm model presets
* feat(setup): remove llm model setup flag
* chore(setup): update llm preset guidance
* docs(setup): document llm model presets
* chore(release): sync uv.lock to 0.9.0
* fix(cli): make sl query --execute work on secret-backed connections
sl query --execute used a parallel SQL executor (createDefaultLocalQueryExecutor)
that passed connection.url verbatim into pg, so file:/env: secret references
failed with "SASL: SCRAM-SERVER-FIRST-MESSAGE: client password must be a string".
Collapse onto the connector-based executor already used by MCP and ingest
(createKtxCliIngestQueryExecutor), which resolves secret references and supports
every driver. Delete the now-dead local/postgres/sqlite query executors, their
tests, and the orphaned hasLocalQueryExecutor driver flag.
* docs(agents): require one implementation per capability
Add a design-reasoning default and a matching self-check question telling agents
to route callers through a single shared implementation of a capability rather
than forking a parallel one, and to fix the shared layer rather than patch one
branch. Encodes the lesson from a divergent SQL-execution-path bug, stated
generally.
CLAUDE.md is a symlink to AGENTS.md, so both agent-instruction files are covered.
* feat(mysql): implement columnStats using INFORMATION_SCHEMA.STATISTICS
Enable column cardinality statistics for the MySQL connector by querying
INFORMATION_SCHEMA.STATISTICS, which provides index-based cardinality
estimates without requiring additional permissions.
- Add generateColumnStatisticsQuery() to KtxMysqlDialect
- Add getColumnStatistics() and columnStats() to KtxMysqlScanConnector
- Flip columnStats capability from false to true
- Add MysqlStatsRow and KtxMysqlColumnStatisticsResult interfaces
- Add tests for dialect query generation and connector stats retrieval
- Update dialect conformance fixture for mysql
* fix(mysql): filter to leading index columns to avoid inflated cardinality
Add AND SEQ_IN_INDEX = 1 to INFORMATION_SCHEMA.STATISTICS query to
ensure only leading index columns are returned. For composite indexes,
non-leading columns report the cardinality of the index prefix rather
than the column's own distinct count, which inflates distinctCount.
Add regression test asserting SEQ_IN_INDEX = 1 is present in the query.
* fix: add trailing newline to dialect.test.ts
---------
Co-authored-by: Andrey Avtomonov <andreybavt@gmail.com>
* feat(cli): define full warehouse dialect contract
* test(cli): keep dialect edge tests focused
* fix(cli): stabilize dialect contract foundation
* refactor(connectors): own read-only query preparation
* refactor(connectors): resolve dialects through registry
* refactor(connectors): keep concrete dialect classes internal
* chore(workspace): enforce dialect import boundary
* refactor(cli): resolve relationship dialect at scan boundary
* refactor(cli): use dialect display parsing for entity details
* refactor(cli): use dialect display parsing for warehouse catalog
* refactor(cli): use dialect SQL in relationship workflows
* test(cli): verify solid dialect scan workflow closure
* test: split cli tests from source tree
* refactor(cli): standardize BigQuery scope listing
* feat(sqlite): implement connector scope listing
* test(connectors): cover required table listing
* feat(cli): add warehouse driver registry
* refactor(setup): route scope discovery through driver registry
* refactor(cli): route local query execution through driver registry
* refactor(historic-sql): route dialect support through driver registry
* refactor(cli): test warehouse connections through driver registry
* fix(cli): close driver registry type export gaps
* Improve setup daemon diagnostics
* refactor(setup): centralize rail-prefixed diagnostics + query-history fallback
Extract errorMessage, writePrefixedLines, and flushPrefixedBufferedCommandOutput
into clack.ts so the setup wizard, managed daemons, and embedding/agent steps
share one rail-formatted writer. setup-databases.ts also adds a
"disable query history and retry" option when the schema-context build fails
and query history is the likely culprit, surfaced via a new
failed-query-history-unavailable status.
* fix(cli): carry catalog through the picker so BigQuery/Snowflake/SQL Server scope filters match
The setup picker's KtxTableListEntry was a 2-level { schema, name }, so
qualifiedTableId always wrote db.name into enabled_tables. When BigQuery,
Snowflake, or SQL Server later ran fast ingest, their introspect step filtered
the scope set with scopedTableNames(scope, { catalog: projectId|database, db })
— catalog was non-null on the introspect side but null in the scope refs, so
every entry was rejected, the live-database adapter staged zero table files,
and detect() failed with 'Adapter "live-database" did not recognize fetched
source output'.
Align the picker boundary with the canonical 3-level KtxTableRef:
- Add catalog: string | null to KtxTableListEntry.
- BigQuery/Snowflake/SQL Server listTables populate catalog from the
resolved projectId / database; Postgres/MySQL/ClickHouse/SQLite set null.
- qualifiedTableId emits catalog.schema.name when catalog is non-null
(resolveEnabledTables already accepts the 3-part shape) and
schemasFromEnabledTables now goes through parseDottedTableEntry so it
recovers the schema correctly from both 2-part and 3-part entries.
- Export parseDottedTableEntry from enabled-tables.ts (@internal) for picker
reuse.
Update listTables expectations in all seven connector tests and the setup /
picker test fixtures. Add a picker regression test that covers the
catalog-bearing round-trip (save + refine).
* fix(cli): allow debug telemetry under opt-out env
