From 28b5e2a83e543ac2b215f2e07f44fbdcf30c5139 Mon Sep 17 00:00:00 2001 From: Andrey Avtomonov Date: Thu, 14 May 2026 00:57:51 +0200 Subject: [PATCH 1/2] fix: align KTX agent tools and repair handling (#73) --- .../memory_agent_bundle_ingest_reconcile.md | 2 +- .../context/skills/ingest_triage/SKILL.md | 4 +-- .../context/skills/lookml_ingest/SKILL.md | 2 +- .../context/skills/notion_synthesize/SKILL.md | 4 +-- packages/context/skills/sl/SKILL.md | 27 +++++++------- packages/context/skills/sl_capture/SKILL.md | 36 ++++++++++++------- packages/context/skills/wiki_capture/SKILL.md | 4 +-- .../src/agent/agent-runner.service.test.ts | 4 +++ .../context/src/agent/agent-runner.service.ts | 3 ++ .../src/ingest/ingest-bundle.runner.test.ts | 10 ++++-- .../src/ingest/stages/stage-index.types.ts | 4 ++- .../emit-reconciliation-records.tool.test.ts | 24 +++++++++++++ .../tools/emit-unmapped-fallback.tool.ts | 6 ++++ .../ingest/tools/eviction-list.tool.test.ts | 2 +- .../src/ingest/tools/eviction-list.tool.ts | 2 +- .../ingest/tools/verification-ledger.tool.ts | 2 +- packages/context/src/llm/generation.ts | 18 ++++++++++ .../context/src/sl/tools/sl-discover.tool.ts | 2 +- .../src/sl/tools/sl-read-source.tool.ts | 2 +- 19 files changed, 113 insertions(+), 45 deletions(-) diff --git a/packages/context/prompts/memory_agent_bundle_ingest_reconcile.md b/packages/context/prompts/memory_agent_bundle_ingest_reconcile.md index 515fecd3..3813afcb 100644 --- a/packages/context/prompts/memory_agent_bundle_ingest_reconcile.md +++ b/packages/context/prompts/memory_agent_bundle_ingest_reconcile.md @@ -12,7 +12,7 @@ Parsimonious. Stage 3 WUs already loaded `ingest_triage` and handled conflicts t 3. If the system prompt includes ``, apply those pins before flagging a same-name or near-duplicate conflict. A pinned `canonicalArtifactKey` keeps the contested name when it is present in the Stage Index; competing variants keep or receive disambiguated names. 4. Sweep both exact-key conflicts and near-duplicate writes. Compare WUs that wrote overlapping SL source names, overlapping wiki keys, the same `tables:` or `sl_refs:` action details, or obviously equivalent topic titles under different wiki keys. Call `stage_diff` to see the actual difference, and use `wiki_read`/`sl_read_source` when two different keys appear to describe the same table, metric, or source-of-truth mapping. If they're the same content, leave one canonical artifact and record the duplicate as subsumed. If they differ per `ingest_triage` rules, apply the correct resolution (rename + capture; election of canonical; silent replace for expression-only re-ingest change; or pinned canonical), then call `emit_conflict_resolution` with the artifact key and decision. 5. For any `wiki_write`, `wiki_remove`, `sl_write_source`, or `sl_edit_source` call you make during reconciliation, include `rawPaths` with only the raw paths that directly caused that reconciliation action. -6. Call `eviction_list()` for deleted raw paths. For each listed artifact, remove it (`sl_delete`, `wiki_remove`) and include the evicted raw path in `rawPaths`. Then call `emit_eviction_decision` with `action: "removed"` for every removed artifact. +6. Call `eviction_list()` for deleted raw paths. For each listed artifact, remove it (`sl_write_source`/`sl_edit_source` with `delete: true` for SL sources, `wiki_remove` for wiki pages) and include the evicted raw path in `rawPaths`. Then call `emit_eviction_decision` with `action: "removed"` for every removed artifact. 7. If the Stage 4 sweep discovers a raw file whose only honest outcome is standalone SQL, wiki-only capture, or a human flag, call `emit_unmapped_fallback` with the raw path, reason, and fallback kind. 8. Use `read_raw_span` to zoom into specific raw files when you need to resolve what two contested measures or wiki pages actually describe. 9. Exit when you've processed every item. diff --git a/packages/context/skills/ingest_triage/SKILL.md b/packages/context/skills/ingest_triage/SKILL.md index df13ed83..c7cee225 100644 --- a/packages/context/skills/ingest_triage/SKILL.md +++ b/packages/context/skills/ingest_triage/SKILL.md @@ -7,7 +7,7 @@ callers: [memory_agent] # Ingest Triage — conflict classification and resolution This skill is loaded in two contexts: -- By a Stage 3 WorkUnit agent when `sl_discover` or an `sl_discover` reveals that a prior WU (or a prior sync) already wrote something that overlaps with what the current WU is about to write. +- By a Stage 3 WorkUnit agent when `sl_discover` reveals that a prior WU (or a prior sync) already wrote something that overlaps with what the current WU is about to write. - By the Stage 4 reconciliation agent for cross-WU sweeps and for eviction decisions. Apply the rules below before every write that could collide with an existing artifact. @@ -32,7 +32,7 @@ Apply the rules below before every write that could collide with an existing art | Definitional contradiction | Same name, substantively different formulas (different aggregation, different filters, different columns) | **Rename + capture**: disambiguate ALL variants with suffix derived from the domain (`churn_risk_engagement_based`, `churn_risk_billing_based`) and write a unified wiki page listing every variant with provenance. The contested name does NOT land in the SL. **Always flag.** | 5. **Eviction (Stage 4 only)**: for each entry in `eviction_list()`: - - Remove the artifact (`sl_delete` for SL sources, `wiki_remove` for wiki pages). + - Remove the artifact (`sl_write_source` or `sl_edit_source` with `delete: true` for SL sources, `wiki_remove` for wiki pages). - Record the removal with `emit_eviction_decision` and `action: "removed"`. ## Why same-ingest vs re-ingest differs diff --git a/packages/context/skills/lookml_ingest/SKILL.md b/packages/context/skills/lookml_ingest/SKILL.md index 5a9c79a3..6a01a355 100644 --- a/packages/context/skills/lookml_ingest/SKILL.md +++ b/packages/context/skills/lookml_ingest/SKILL.md @@ -84,7 +84,7 @@ SL source, `tables:` frontmatter, `sl_refs`, or `emit_unmapped_fallback`: **Required flow before writing any overlay or standalone**: -1. Call `sl_discover()` for each base table you're about to touch. That returns the real columns. +1. Call `sl_discover({ query: "" })` for each base table you're about to touch. That returns the real columns. 2. If the table isn't in the manifest, use the warehouse `connectionName` returned by `discover_data` or the target connection chosen from `sl_discover`, then call a dialect-appropriate SQL probe with that diff --git a/packages/context/skills/notion_synthesize/SKILL.md b/packages/context/skills/notion_synthesize/SKILL.md index 524c6832..1b5417e3 100644 --- a/packages/context/skills/notion_synthesize/SKILL.md +++ b/packages/context/skills/notion_synthesize/SKILL.md @@ -20,7 +20,7 @@ Each WorkUnit is either a single Notion page/span or a topical cluster of relate 4. Use `context_evidence_search`, `context_evidence_read`, and `context_evidence_neighbors` to pull supporting chunks when indexed evidence is relevant. Pass `chunkId` and `documentId` values verbatim as returned by the evidence tools. 5. Write durable business knowledge with `wiki_write`. Aim for a small number of high-quality pages per WorkUnit or cluster. Include `rawPaths` with the exact Notion raw files that support each page. 6. When the Notion content defines a reusable dataset, metric, segment, join rule, source-of-truth mapping, or table with explicit columns, load `sl_capture`, discover existing sources first with `sl_discover` or `sl_read_source`, then use `sl_write_source` or `sl_edit_source` only for a confirmed mapped non-Notion target source. Include `rawPaths` with the exact Notion raw files that support the SL action. If no mapped target exists, call `emit_unmapped_fallback` and keep the content wiki-only. -7. For every deleted raw path in the Eviction Set, call `eviction_list`, decide retention, then `context_eviction_decision_write`. Do this even when no wiki write is needed. +7. For every deleted raw path in the Eviction Set, call `eviction_list`, decide retention, then `emit_eviction_decision`. Do this even when no wiki write is needed. ## What To Capture @@ -99,6 +99,6 @@ SL source, `tables:` frontmatter, `sl_refs`, or `emit_unmapped_fallback`: ## Tools -Allowed: `read_raw_file`, `read_raw_span`, `wiki_search`, `wiki_read`, `wiki_write`, `discover_data`, `entity_details`, `sql_execution`, `sl_discover`, `sl_read_source`, `sl_write_source`, `sl_edit_source`, `sl_validate`, `context_evidence_search`, `context_evidence_read`, `context_evidence_neighbors`, `emit_unmapped_fallback`, `eviction_list`, `context_eviction_decision_write`. +Allowed: `read_raw_file`, `read_raw_span`, `wiki_search`, `wiki_read`, `wiki_write`, `discover_data`, `entity_details`, `sql_execution`, `sl_discover`, `sl_read_source`, `sl_write_source`, `sl_edit_source`, `sl_validate`, `context_evidence_search`, `context_evidence_read`, `context_evidence_neighbors`, `emit_unmapped_fallback`, `eviction_list`, `emit_eviction_decision`. Not allowed: `context_candidate_write`, `context_candidate_mark`. diff --git a/packages/context/skills/sl/SKILL.md b/packages/context/skills/sl/SKILL.md index f7077c33..7103a276 100644 --- a/packages/context/skills/sl/SKILL.md +++ b/packages/context/skills/sl/SKILL.md @@ -1,6 +1,6 @@ --- name: sl -description: KTX's semantic layer — a structured catalog of sources (tables/views), measures, joins, and segments expressed as YAML. Covers the schema and how to query it via `semantic_query`. Use when the task involves querying pre-defined metrics (ARR, churn, retention, LTV, MAU) or reading SL source YAML to understand the catalog. Capture is handled by the `sl_capture` skill (memory-agent only). +description: KTX's semantic layer — a structured catalog of sources (tables/views), measures, joins, and segments expressed as YAML. Covers the schema and how to query it via `sl_query`. Use when the task involves querying pre-defined metrics (ARR, churn, retention, LTV, MAU) or reading SL source YAML to understand the catalog. Capture is handled by the `sl_capture` skill (memory-agent only). --- # Semantic Layer @@ -9,7 +9,7 @@ KTX's semantic layer (SL) is a structured catalog. Each **source** represents a This skill covers two parts: - **Part 1** — Schema reference (what an SL source looks like). -- **Part 2** — Querying via `semantic_query`. +- **Part 2** — Querying via `sl_query`. Capture (when and how to add new patterns to the SL) is a separate concern handled by the memory-agent — see the `sl_capture` skill if you are running in capture mode. The research agent **reads** and **queries** the SL via the tools described here; it does not write to it. @@ -162,7 +162,7 @@ segments: description: Orders that were paid and not refunded ``` -Named, reusable boolean predicates scoped to one source. Reference by bare name in a measure's `segments: []`, or by dotted form `source.segment_name` in a `semantic_query`. Segments are predicates only — they are NOT selectable as dimensions. If you need to group by the predicate, add a `columns[]` entry instead. +Named, reusable boolean predicates scoped to one source. Reference by bare name in a measure's `segments: []`, or by dotted form `source.segment_name` in an `sl_query`. Segments are predicates only — they are NOT selectable as dimensions. If you need to group by the predicate, add a `columns[]` entry instead. ### Cross-references with the wiki @@ -170,11 +170,11 @@ The reverse edge (wiki pages that cite this source) is derived automatically fro --- -## Part 2 — Querying via `semantic_query` +## Part 2 — Querying via `sl_query` -The `semantic_query` tool generates correct SQL from a structured query. It handles joins, fan-out prevention, aggregation correctness, and filter classification automatically. Prefer it over writing raw SQL whenever the SL has the relevant sources. +The `sl_query` tool generates correct SQL from a structured query. It handles joins, fan-out prevention, aggregation correctness, and filter classification automatically. Prefer it over writing raw SQL whenever the SL has the relevant sources. -### When to prefer semantic_query over raw SQL +### When to prefer sl_query over raw SQL - A pre-defined measure already exists (`source.measure_name` appears in the catalog). - The question combines fields from multiple sources — the engine resolves the join path automatically. @@ -189,15 +189,12 @@ Use raw SQL (`sql_execution`) only when: ```json { "connectionId": "uuid-of-the-connection", - "reasoning": "Brief note on what this query analyzes", - "query": { - "measures": ["orders.total_revenue", "sum(orders.amount)"], - "dimensions": ["customers.segment", { "field": "orders.created_at", "granularity": "month" }], - "filters": ["orders.status != 'cancelled'", "orders.total_revenue > 10000"], - "segments": ["orders.paid_non_refunded"], - "order_by": [{ "field": "orders.created_at", "direction": "desc" }], - "limit": 1000 - } + "measures": ["orders.total_revenue", "sum(orders.amount)"], + "dimensions": ["customers.segment", { "field": "orders.created_at", "granularity": "month" }], + "filters": ["orders.status != 'cancelled'", "orders.total_revenue > 10000"], + "segments": ["orders.paid_non_refunded"], + "order_by": [{ "field": "orders.created_at", "direction": "desc" }], + "limit": 1000 } ``` diff --git a/packages/context/skills/sl_capture/SKILL.md b/packages/context/skills/sl_capture/SKILL.md index a40111ea..4ec21545 100644 --- a/packages/context/skills/sl_capture/SKILL.md +++ b/packages/context/skills/sl_capture/SKILL.md @@ -63,7 +63,7 @@ Preferred: - name: total_revenue expr: sum(amount) ``` -Callers filter `region = 'US'` at `semantic_query` time. +Callers filter `region = 'US'` at query time. **Bake constants in only when the filter has named business meaning that won't change** (`enterprise_arr` for a contractually defined tier), cannot be expressed via the source's dimensions, or comes from a regulated/fixed list. @@ -100,7 +100,7 @@ measures: **Extract repeated filter bundles into named segments.** If the same predicate appears on multiple measures of the same source, lift it to a `segments[]` entry and have each measure reference it. One edit updates every measure that depends on it. -**Never write a standalone file on a manifest-backed name.** If `sl_discover({ tableName })` finds an existing schema for that name, you MUST write an overlay (`name:` + `measures:`/`segments:`/`descriptions:` only — no `sql:`, `table:`, `grain:`, `columns:`, `joins:`). A standalone with `sql:` or `table:` on a manifest-backed name clobbers the inherited columns and joins; `sl_write_source` and `sl_validate` both reject this shape with a clear fix hint. Always run `sl_discover` before your first write on any existing name. +**Never write a standalone file on a manifest-backed name.** If `sl_discover({ query: "" })` finds an existing schema for that name, you MUST write an overlay (`name:` + `measures:`/`segments:`/`descriptions:` only — no `sql:`, `table:`, `grain:`, `columns:`, `joins:`). A standalone with `sql:` or `table:` on a manifest-backed name clobbers the inherited columns and joins; `sl_write_source` and `sl_validate` both reject this shape with a clear fix hint. Always run `sl_discover` before your first write on any existing name. **Prefer overlay decomposition over standalone SQL sources.** Before reaching for `source_type: sql`, check whether the metric decomposes into measures on existing overlays (including cross-source derived measures). Use `source_type: sql` only when: - The metric requires per-user/per-entity derivation that cannot be expressed as a single `expr` (e.g., `EXISTS` over a time-windowed subset), OR @@ -209,10 +209,10 @@ SL source, `tables:` frontmatter, `sl_refs`, or `emit_unmapped_fallback`: ## Tool sequence 1. `sl_discover` — see what source files exist. -2. `sl_discover({ tableName })` — **REQUIRED before the first write on any name**. Shows columns/joins/grain from the manifest. If the call returns a schema, you MUST write an overlay, not a standalone. Skipping this is the #1 cause of accidentally shadowing the manifest. -3. `sl_read_source({ sourceName })` — read the raw YAML before editing. -4. For modifications: `sl_edit_source({ sourceName, old_string, new_string })` with exact-string replacements. `old_string` must match exactly and be unique in the file. -5. For new sources or full rewrites: `sl_write_source({ sourceName, content })` with the full YAML content. +2. `sl_discover({ query: "" })` — **REQUIRED before the first write on any name**. Shows columns/joins/grain from the manifest. If the call returns a schema, you MUST write an overlay, not a standalone. Skipping this is the #1 cause of accidentally shadowing the manifest. +3. `sl_read_source({ connectionId, sourceName })` — read the raw YAML before editing. +4. For modifications: `sl_edit_source({ connectionId, sourceName, yaml_edits: [{ oldText, newText, reason }] })` with exact-string replacements. `oldText` must match exactly and be unique in the file. +5. For new sources or full rewrites: `sl_write_source({ connectionId, sourceName, source })` with the full structured source definition. 6. For join discovery: use `sql_execution({connectionName: "warehouse", sql: "SELECT count(*) FROM public.orders o JOIN public.customers c ON c.id = o.customer_id LIMIT 20"})` with the target warehouse connection name and dialect-correct table names to verify the join key exists in both tables and assess cardinality before declaring the join. 7. Cross-reference knowledge: author the edge once on the **wiki** side via `sl_refs: [source_name]` in the page's front-matter. The reverse edge (wiki pages that cite an SL source) is derived automatically by the reconciler — do not add a `knowledge_refs:` field to SL YAMLs. 8. `sl_validate` — run after writing or editing to surface schema issues, duplicate measure names, and cross-source validation errors. Read-only; the writes are already committed (the squash-at-end flow will collapse them into one commit). @@ -235,13 +235,21 @@ Existing index: `orders [measures=0, joins=0] — candidate for enrichment`. ``` sl_discover() → orders.yaml does not exist yet -sl_discover({ tableName: "orders" }) +sl_discover({ query: "orders" }) → see grain, columns, no current overlay sl_write_source({ + connectionId: "warehouse", sourceName: "orders", - content: "name: orders\nmeasures:\n - name: avg_order_value\n expr: avg(amount)\n description: Mean order transaction amount — filter by product_category at query time\n" + source: { + name: "orders", + measures: [{ + name: "avg_order_value", + expr: "avg(amount)", + description: "Mean order transaction amount - filter by product_category at query time" + }] + } }) -sl_validate() +sl_validate({ connectionId: "warehouse" }) → clean ``` @@ -258,16 +266,17 @@ Current user: "Wait, by 'active' I mean users who have placed an order in the la The existing `users.active_count` measure is wrong by the new definition. ``` -sl_read_source({ sourceName: "users" }) +sl_read_source({ connectionId: "warehouse", sourceName: "users" }) → see the wrong measure sl_edit_source({ + connectionId: "warehouse", sourceName: "users", yaml_edits: [{ oldText: " - name: active_count\n expr: \"count(*)\"\n filter: \"last_login_at > now() - interval '30 days'\"\n description: Users who logged in within the last 30 days", newText: " - name: active_count\n expr: \"count(distinct case when last_order_at > now() - interval '30 days' then user_id end)\"\n description: Users with at least one order in the last 30 days" }] }) -sl_validate() +sl_validate({ connectionId: "warehouse" }) ``` If you only added a new measure, the old incorrect `active_count` would stay and future queries would keep answering the wrong question. @@ -277,7 +286,7 @@ If you only added a new measure, the old incorrect `active_count` would stay and Prior turn: user asked to correlate LTV with protocol count; assistant joined `fct_orders` with `fct_mau_multiprotocol` on `admin_user_id` in raw SQL. ``` -sl_read_source({ sourceName: "fct_orders" }) +sl_read_source({ connectionId: "warehouse", sourceName: "fct_orders" }) → no joins section yet sql_execution({ connectionName: "warehouse", @@ -285,13 +294,14 @@ sql_execution({ }) → confirms cardinality (many orders per MAU row = many_to_one) sl_edit_source({ + connectionId: "warehouse", sourceName: "fct_orders", yaml_edits: [{ oldText: "measures:", newText: "joins:\n - to: fct_mau_multiprotocol\n on: admin_user_id = fct_mau_multiprotocol.admin_user_id\n relationship: many_to_one\nmeasures:" }] }) -sl_validate() +sl_validate({ connectionId: "warehouse" }) ``` Always verify joins with `sql_execution` before adding them. diff --git a/packages/context/skills/wiki_capture/SKILL.md b/packages/context/skills/wiki_capture/SKILL.md index 30188be6..d57a39ad 100644 --- a/packages/context/skills/wiki_capture/SKILL.md +++ b/packages/context/skills/wiki_capture/SKILL.md @@ -31,7 +31,7 @@ Do NOT capture: - Temporary instructions scoped to the current chat. - Ad-hoc formatting preferences. - Information already present in the semantic layer (column names, join paths, measure formulas — those belong in SL). -- **Query results, snapshots, or time-bounded benchmark tables.** Numbers go stale; pasting "Oct 2025: 25%, Nov 2025: 19.9%, …" creates misinformation as soon as new data lands. Reference the SL source by name (`sl_refs`) and let future queries pull live data — the wiki captures the *rule* (definition, exclusion, segmentation), the SL source captures the *measure*, and `semantic_query` captures the *current values*. +- **Query results, snapshots, or time-bounded benchmark tables.** Numbers go stale; pasting "Oct 2025: 25%, Nov 2025: 19.9%, …" creates misinformation as soon as new data lands. Reference the SL source by name (`sl_refs`) and let future query tools pull live data — the wiki captures the *rule* (definition, exclusion, segmentation), the SL source captures the *measure*, and query execution captures the *current values*. - **Interpretive narrative tied to a specific snapshot** ("M1 retention degraded sharply from Dec 2025"). The observation is anchored to data that will move; the actionable convention (e.g., "always exclude in-progress cohorts") may be worth capturing on its own, but the snapshot-specific commentary is not. If nothing is worth capturing, respond without calling any tool. @@ -136,7 +136,7 @@ wiki_search({ query: "refund revenue paid orders" }) → returns `revenue-definition` (related — defines paid-orders filter) sl_discover({ query: "refund rate" }) → returns fct_orders (score 0.08), fct_gaap_revenue (0.06) -sl_read_source({ sourceName: "fct_orders" }) +sl_read_source({ connectionId: "warehouse", sourceName: "fct_orders" }) → confirms amount_refunded_dollars and transaction_amount_dollars exist wiki_write({ key: "refund-rate-definition", diff --git a/packages/context/src/agent/agent-runner.service.test.ts b/packages/context/src/agent/agent-runner.service.test.ts index 70b4e0da..8f405841 100644 --- a/packages/context/src/agent/agent-runner.service.test.ts +++ b/packages/context/src/agent/agent-runner.service.test.ts @@ -40,6 +40,8 @@ describe('AgentRunnerService.runLoop', () => { it('passes systemPrompt, userPrompt, tools, and step budget through to generateText', async () => { (generateText as any).mockResolvedValue({ text: 'ok', toolCalls: [], steps: [] }); + const repairHandler = vi.fn(); + llmProvider.repairToolCallHandler.mockReturnValueOnce(repairHandler); const tools = { noop: { description: 'noop', inputSchema: {}, execute: vi.fn() } }; await runner.runLoop({ modelRole: 'candidateExtraction', @@ -59,7 +61,9 @@ describe('AgentRunnerService.runLoop', () => { expect(call.tools).toEqual(tools); expect(call.stopWhen).toBe(17); expect(call.temperature).toBe(0); + expect(call.experimental_repairToolCall).toBe(repairHandler); expect(llmProvider.getModel).toHaveBeenCalledWith('candidateExtraction'); + expect(llmProvider.repairToolCallHandler).toHaveBeenCalledWith({ source: 'ktx-agent-runner' }); }); it('returns stopReason=natural when the loop completes without error', async () => { diff --git a/packages/context/src/agent/agent-runner.service.ts b/packages/context/src/agent/agent-runner.service.ts index c394fd75..92daad8f 100644 --- a/packages/context/src/agent/agent-runner.service.ts +++ b/packages/context/src/agent/agent-runner.service.ts @@ -73,6 +73,9 @@ export class AgentRunnerService { temperature: 0, stopWhen: stepCountIs(params.stepBudget), experimental_telemetry: this.deps.telemetry?.createTelemetry(params.telemetryTags), + experimental_repairToolCall: this.deps.llmProvider.repairToolCallHandler({ + source: params.telemetryTags.operationName ?? 'ktx-agent-runner', + }), messages: built.messages, tools: built.tools as Record, onStepFinish: async () => { diff --git a/packages/context/src/ingest/ingest-bundle.runner.test.ts b/packages/context/src/ingest/ingest-bundle.runner.test.ts index b9831c0f..bc25308f 100644 --- a/packages/context/src/ingest/ingest-bundle.runner.test.ts +++ b/packages/context/src/ingest/ingest-bundle.runner.test.ts @@ -695,7 +695,8 @@ describe('IngestBundleRunner — Stages 1 → 7', () => { await params.toolSet.emit_unmapped_fallback.execute( { rawPath: 'a.yml', - reason: 'semantic_not_representable', + reason: 'parse_error', + clarification: 'semantic_not_representable', fallback: 'flagged', }, { toolCallId: 'fallback-1', messages: [] }, @@ -954,6 +955,7 @@ describe('IngestBundleRunner — Stages 1 → 7', () => { { rawPath: 'a.yml', reason: 'conversion_metric_unsupported', + detail: expect.stringContaining('conversion metric'), fallback: 'flagged', }, ], @@ -1006,7 +1008,8 @@ describe('IngestBundleRunner — Stages 1 → 7', () => { await params.toolSet.emit_unmapped_fallback.execute( { rawPath: 'cards/untranslated.json', - reason: 'metabase_sql_untranslated', + reason: 'parse_error', + clarification: 'metabase_sql_untranslated', fallback: 'flagged', }, { toolCallId: 'fallback-1', messages: [] }, @@ -1053,7 +1056,8 @@ describe('IngestBundleRunner — Stages 1 → 7', () => { unmappedFallbacks: [ { rawPath: 'cards/untranslated.json', - reason: 'metabase_sql_untranslated', + reason: 'parse_error', + detail: expect.stringContaining('metabase_sql_untranslated'), fallback: 'flagged', }, ], diff --git a/packages/context/src/ingest/stages/stage-index.types.ts b/packages/context/src/ingest/stages/stage-index.types.ts index 7de26bc8..fa1a652e 100644 --- a/packages/context/src/ingest/stages/stage-index.types.ts +++ b/packages/context/src/ingest/stages/stage-index.types.ts @@ -37,7 +37,9 @@ export type UnmappedFallbackReason = | 'multiple_table_references' | 'unsupported_dialect' | 'parse_error' - | 'missing_target_table'; + | 'missing_target_table' + | 'cumulative_metric_unsupported' + | 'conversion_metric_unsupported'; export interface UnmappedFallbackRecord { rawPath: string; diff --git a/packages/context/src/ingest/tools/emit-reconciliation-records.tool.test.ts b/packages/context/src/ingest/tools/emit-reconciliation-records.tool.test.ts index 9178c989..1cd77514 100644 --- a/packages/context/src/ingest/tools/emit-reconciliation-records.tool.test.ts +++ b/packages/context/src/ingest/tools/emit-reconciliation-records.tool.test.ts @@ -182,6 +182,30 @@ describe('reconciliation emit tools', () => { ]); }); + it('records MetricFlow-specific unsupported fallback reasons', async () => { + const stageIndex = makeStageIndex(); + const tool = createEmitUnmappedFallbackTool({ + stageIndex, + allowedPaths: new Set(['metrics/conversion.yml']), + }); + + const output = await executeTool(tool, { + rawPath: 'metrics/conversion.yml', + reason: 'conversion_metric_unsupported', + fallback: 'flagged', + }); + + expect(output).toContain('conversion metric'); + expect(stageIndex.unmappedFallbacks).toEqual([ + { + rawPath: 'metrics/conversion.yml', + reason: 'conversion_metric_unsupported', + detail: expect.stringContaining('conversion metric'), + fallback: 'flagged', + }, + ]); + }); + it('rejects unmapped fallback decisions for raw paths outside the allowed set', async () => { const stageIndex = makeStageIndex(); const tool = createEmitUnmappedFallbackTool({ diff --git a/packages/context/src/ingest/tools/emit-unmapped-fallback.tool.ts b/packages/context/src/ingest/tools/emit-unmapped-fallback.tool.ts index 33a8610e..cdb1a483 100644 --- a/packages/context/src/ingest/tools/emit-unmapped-fallback.tool.ts +++ b/packages/context/src/ingest/tools/emit-unmapped-fallback.tool.ts @@ -17,6 +17,8 @@ const unmappedFallbackReasonSchema = z.enum([ 'unsupported_dialect', 'parse_error', 'missing_target_table', + 'cumulative_metric_unsupported', + 'conversion_metric_unsupported', ]); function sameUnmappedFallback(left: UnmappedFallbackRecord, right: UnmappedFallbackRecord): boolean { @@ -47,6 +49,10 @@ function canonicalDetail(reason: UnmappedFallbackReason, tableRef: string | unde return `${tableClause} uses a SQL dialect that is not yet supported.`; case 'parse_error': return `${tableClause} could not be parsed.`; + case 'cumulative_metric_unsupported': + return `${tableClause} is a cumulative metric, which is not yet supported as a first-class semantic-layer primitive.`; + case 'conversion_metric_unsupported': + return `${tableClause} is a conversion metric, which is not yet supported as a first-class semantic-layer primitive.`; } } diff --git a/packages/context/src/ingest/tools/eviction-list.tool.test.ts b/packages/context/src/ingest/tools/eviction-list.tool.test.ts index 1bd1d82a..96fb7f65 100644 --- a/packages/context/src/ingest/tools/eviction-list.tool.test.ts +++ b/packages/context/src/ingest/tools/eviction-list.tool.test.ts @@ -51,6 +51,6 @@ describe('eviction_list tool', () => { deletedRawPaths: [], }); - expect(tool.description).toContain('context_eviction_decision_write'); + expect(tool.description).toContain('emit_eviction_decision'); }); }); diff --git a/packages/context/src/ingest/tools/eviction-list.tool.ts b/packages/context/src/ingest/tools/eviction-list.tool.ts index 4ed08d63..6a34d48f 100644 --- a/packages/context/src/ingest/tools/eviction-list.tool.ts +++ b/packages/context/src/ingest/tools/eviction-list.tool.ts @@ -12,7 +12,7 @@ export interface EvictionListDeps { export function createEvictionListTool(deps: EvictionListDeps) { return tool({ description: - 'List every artifact that the most recent completed sync produced from a now-deleted raw file. Remove each listed artifact and record the decision with context_eviction_decision_write so the ingest report lists every deleted-source decision.', + 'List every artifact that the most recent completed sync produced from a now-deleted raw file. Remove each listed artifact and record the decision with emit_eviction_decision so the ingest report lists every deleted-source decision.', inputSchema: z.object({}), execute: async () => { if (deps.deletedRawPaths.length === 0) { diff --git a/packages/context/src/ingest/tools/verification-ledger.tool.ts b/packages/context/src/ingest/tools/verification-ledger.tool.ts index ac880607..af27b58d 100644 --- a/packages/context/src/ingest/tools/verification-ledger.tool.ts +++ b/packages/context/src/ingest/tools/verification-ledger.tool.ts @@ -28,7 +28,7 @@ const WRITE_TOOL_NAMES = new Set([ ]); export const VERIFICATION_LEDGER_PROMPT = ` -Before any write-capable tool call (wiki_write, wiki_remove, sl_write_source, sl_edit_source, emit_unmapped_fallback), call record_verification_ledger. +Before any durable wiki, semantic-layer, or unmapped-fallback write (wiki_write, wiki_remove, sl_write_source, sl_edit_source, emit_unmapped_fallback), call record_verification_ledger. The ledger is a model-authored checkpoint, not a deterministic parser gate. Summarize the verification protocol from the loaded skill, list identifiers verified with discover_data/entity_details/sql_execution, and list anything intentionally left unverified. If the write contains no warehouse identifiers, say that explicitly. If a write tool returns verification_ledger_required, complete the ledger and retry the write. `; diff --git a/packages/context/src/llm/generation.ts b/packages/context/src/llm/generation.ts index a7afd5b9..1bbdbcab 100644 --- a/packages/context/src/llm/generation.ts +++ b/packages/context/src/llm/generation.ts @@ -4,6 +4,10 @@ import { generateText, Output, type FlexibleSchema, type ToolSet } from 'ai'; type GenerateTextInput = Parameters[0]; type GenerateTextFn = (input: GenerateTextInput) => Promise<{ text?: string; output?: unknown }>; +function hasTools(tools: ToolSet): boolean { + return Object.keys(tools).length > 0; +} + interface GenerateKtxTextInput { llmProvider: KtxLlmProvider; role: KtxModelRole; @@ -30,6 +34,13 @@ export async function generateKtxText(input: GenerateKtxTextInput): Promise( temperature: input.temperature ?? 0, messages: built.messages, tools: built.tools as ToolSet, + ...(hasTools(built.tools as ToolSet) + ? { + experimental_repairToolCall: input.llmProvider.repairToolCallHandler({ + source: `ktx-${input.role}`, + }), + } + : {}), output: Output.object({ schema: input.schema as FlexibleSchema, }), diff --git a/packages/context/src/sl/tools/sl-discover.tool.ts b/packages/context/src/sl/tools/sl-discover.tool.ts index 97426b40..74570c2e 100644 --- a/packages/context/src/sl/tools/sl-discover.tool.ts +++ b/packages/context/src/sl/tools/sl-discover.tool.ts @@ -53,7 +53,7 @@ export class SlDiscoverTool extends BaseSemanticLayerTool Discover available semantic layer sources, columns, measures, and joins. When called without a connectionId, discovers sources across ALL data sources — grouped by data source name and ID. -Use this to understand what data is available before writing a semantic_query. +Use this to understand what data is available before querying through the semantic layer. diff --git a/packages/context/src/sl/tools/sl-read-source.tool.ts b/packages/context/src/sl/tools/sl-read-source.tool.ts index fb5de830..e4602f31 100644 --- a/packages/context/src/sl/tools/sl-read-source.tool.ts +++ b/packages/context/src/sl/tools/sl-read-source.tool.ts @@ -36,7 +36,7 @@ Use this when you need to understand how a source is built — e.g., before edit - To discover what sources/measures/dimensions are available for querying — use sl_discover instead -- To query data — use semantic_query or create_widget with slQuery +- To query data — use the semantic-layer query surface (\`sl_query\` in MCP) `; } From 0a261fe8a47b9fb75b01bbad209e303d35fd6f25 Mon Sep 17 00:00:00 2001 From: Andrey Avtomonov Date: Thu, 14 May 2026 01:13:31 +0200 Subject: [PATCH 2/2] ci: add codecov coverage reporting (#82) * ci: add codecov coverage reporting * ci: fix codecov and secret scan checks * ci: fix smoke and artifact checks --- .github/workflows/ci.yml | 64 +++++++ knip.json | 1 + package.json | 3 + packages/cli/package.json | 1 + packages/cli/src/doctor.test.ts | 6 +- packages/cli/src/setup-sources.test.ts | 4 +- packages/cli/src/standalone-smoke.test.ts | 10 +- packages/connector-bigquery/package.json | 1 + packages/connector-clickhouse/package.json | 1 + packages/connector-mysql/package.json | 1 + packages/connector-postgres/package.json | 1 + packages/connector-snowflake/package.json | 1 + packages/connector-sqlite/package.json | 1 + packages/connector-sqlserver/package.json | 1 + packages/context/package.json | 1 + packages/llm/package.json | 1 + pnpm-lock.yaml | 187 +++++++++++++++++++-- pyproject.toml | 1 + scripts/package-artifacts.mjs | 5 +- uv.lock | 2 + 20 files changed, 272 insertions(+), 21 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ff0b7843..de9d056d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -188,6 +188,70 @@ jobs: - name: Run Python checks run: uv run pytest + coverage: + name: Coverage + runs-on: ubuntu-latest + permissions: + contents: read + id-token: write + steps: + - name: Checkout repository + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + + - name: Setup pnpm + uses: pnpm/action-setup@0e279bb959325dab635dd2c09392533439d90093 # v6.0.8 + with: + run_install: false + + - name: Setup Node.js + uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + with: + node-version: "24" + cache: "pnpm" + cache-dependency-path: "pnpm-lock.yaml" + + - name: Install TypeScript dependencies + run: pnpm install --frozen-lockfile + + - name: Setup Python + uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 + with: + python-version: "3.13" + + - name: Setup uv + uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 + with: + version: "0.11.11" + enable-cache: true + cache-dependency-glob: "uv.lock" + + - name: Install Python dependencies + run: uv sync --all-packages --all-groups + + - name: Generate TypeScript coverage + run: pnpm run test:coverage:ts + + - name: Upload TypeScript coverage + uses: codecov/codecov-action@75cd11691c0faa626561e295848008c8a7dddffe # v5.5.4 + with: + use_oidc: true + files: ./packages/cli/coverage/lcov.info,./packages/connector-bigquery/coverage/lcov.info,./packages/connector-clickhouse/coverage/lcov.info,./packages/connector-mysql/coverage/lcov.info,./packages/connector-postgres/coverage/lcov.info,./packages/connector-snowflake/coverage/lcov.info,./packages/connector-sqlite/coverage/lcov.info,./packages/connector-sqlserver/coverage/lcov.info,./packages/context/coverage/lcov.info,./packages/llm/coverage/lcov.info + flags: typescript + name: typescript + fail_ci_if_error: false + + - name: Generate Python coverage + run: pnpm run test:coverage:py + + - name: Upload Python coverage + uses: codecov/codecov-action@75cd11691c0faa626561e295848008c8a7dddffe # v5.5.4 + with: + use_oidc: true + files: ./coverage/python.xml + flags: python + name: python + fail_ci_if_error: false + artifact-checks: name: Artifact checks runs-on: ubuntu-latest diff --git a/knip.json b/knip.json index a6bf5d26..83a8fb1d 100644 --- a/knip.json +++ b/knip.json @@ -79,6 +79,7 @@ "**/*.gen.ts", "**/*.generated.ts" ], + "ignoreBinaries": ["uv"], "ignoreIssues": { "packages/cli/src/clack.ts": ["exports"], "packages/cli/src/commands/connection-metabase-setup.ts": ["exports", "types"], diff --git a/package.json b/package.json index b105784b..5fa920ff 100644 --- a/package.json +++ b/package.json @@ -36,6 +36,9 @@ "relationships:verify-orbit": "node scripts/relationship-orbit-verification.mjs", "smoke": "pnpm run build && pnpm --filter @ktx/cli run smoke", "test": "node --test scripts/*.test.mjs && pnpm --filter './packages/*' run test", + "test:coverage": "pnpm run test:coverage:ts && pnpm run test:coverage:py", + "test:coverage:py": "uv run pytest --cov=python/ktx-sl/semantic_layer --cov=python/ktx-daemon/src/ktx_daemon --cov-report=xml:coverage/python.xml --cov-report=term", + "test:coverage:ts": "pnpm --filter './packages/*' run build && pnpm --filter './packages/*' run test --coverage --coverage.reporter=lcov --coverage.exclude='dist/**'", "test:slow": "pnpm --filter @ktx/context run test:slow && pnpm --filter @ktx/cli run test:slow", "type-check": "pnpm --filter './packages/*' run type-check" }, diff --git a/packages/cli/package.json b/packages/cli/package.json index 19bcc64b..65895f89 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -54,6 +54,7 @@ "@types/better-sqlite3": "^7.6.13", "@types/node": "^25.7.0", "@types/react": "^19.2.14", + "@vitest/coverage-v8": "^4.1.6", "better-sqlite3": "^12.10.0", "ink-testing-library": "^4.0.0", "typescript": "^6.0.3", diff --git a/packages/cli/src/doctor.test.ts b/packages/cli/src/doctor.test.ts index d5db8ab6..b96ee04a 100644 --- a/packages/cli/src/doctor.test.ts +++ b/packages/cli/src/doctor.test.ts @@ -281,7 +281,7 @@ describe('runKtxDoctor', () => { }); it('runs project checks against a valid ktx.yaml', async () => { - process.env.ANTHROPIC_API_KEY = 'test-key'; + process.env.ANTHROPIC_API_KEY = 'test-key'; // pragma: allowlist secret await writeFile( join(tempDir, 'ktx.yaml'), [ @@ -306,7 +306,7 @@ describe('runKtxDoctor', () => { ].join('\n'), 'utf-8', ); - process.env.OPENAI_API_KEY = 'test-key'; + process.env.OPENAI_API_KEY = 'test-key'; // pragma: allowlist secret const testIo = makeIo(); await expect( @@ -357,7 +357,7 @@ describe('runKtxDoctor', () => { }); it('warns when semantic-search embeddings are not configured', async () => { - process.env.ANTHROPIC_API_KEY = 'test-key'; + process.env.ANTHROPIC_API_KEY = 'test-key'; // pragma: allowlist secret await writeFile( join(tempDir, 'ktx.yaml'), [ diff --git a/packages/cli/src/setup-sources.test.ts b/packages/cli/src/setup-sources.test.ts index 0a0eab2c..cb21cd20 100644 --- a/packages/cli/src/setup-sources.test.ts +++ b/packages/cli/src/setup-sources.test.ts @@ -1108,13 +1108,13 @@ describe('setup sources step', () => { }); expect(discoverMetabaseDatabases).toHaveBeenCalledWith({ sourceUrl: 'https://metabase-new.example.com', - sourceApiKeyRef: 'env:METABASE_API_KEY', + sourceApiKeyRef: 'env:METABASE_API_KEY', // pragma: allowlist secret sourceConnectionId: 'metabase-main', }); expect((await readConfig()).connections['metabase-main']).toMatchObject({ driver: 'metabase', api_url: 'https://metabase-new.example.com', - api_key_ref: 'env:METABASE_API_KEY', + api_key_ref: 'env:METABASE_API_KEY', // pragma: allowlist secret mappings: { databaseMappings: { '2': 'warehouse' }, syncEnabled: { '2': true }, diff --git a/packages/cli/src/standalone-smoke.test.ts b/packages/cli/src/standalone-smoke.test.ts index 0f53cca0..ee1b0150 100644 --- a/packages/cli/src/standalone-smoke.test.ts +++ b/packages/cli/src/standalone-smoke.test.ts @@ -200,9 +200,13 @@ describe('standalone built ktx CLI smoke', () => { it('runs doctor setup through the built binary', async () => { const result = await runBuiltCli(['status', '--no-input']); - expect(result.stdout).toMatch(/KTX (setup|project) doctor/); - expect(result.stdout).toContain('Node 22+'); - expect(result.stdout).toContain('Workspace-local CLI'); + expect(result.stdout).toMatch(/KTX (setup doctor|project doctor|status)/); + if (result.stdout.includes('No project here yet.')) { + expect(result.stdout).toContain('Before you can run ktx setup'); + } else { + expect(result.stdout).toContain('Node 22+'); + expect(result.stdout).toContain('Workspace-local CLI'); + } expect(result.stderr === '' || result.stderr.startsWith('Project: ')).toBe(true); expect([0, 1]).toContain(result.code); }); diff --git a/packages/connector-bigquery/package.json b/packages/connector-bigquery/package.json index c21c332f..591e5193 100644 --- a/packages/connector-bigquery/package.json +++ b/packages/connector-bigquery/package.json @@ -31,6 +31,7 @@ }, "devDependencies": { "@types/node": "^25.7.0", + "@vitest/coverage-v8": "^4.1.6", "typescript": "^6.0.3", "vitest": "^4.1.6" }, diff --git a/packages/connector-clickhouse/package.json b/packages/connector-clickhouse/package.json index 8cfbb8eb..568ed6b2 100644 --- a/packages/connector-clickhouse/package.json +++ b/packages/connector-clickhouse/package.json @@ -31,6 +31,7 @@ }, "devDependencies": { "@types/node": "^25.7.0", + "@vitest/coverage-v8": "^4.1.6", "typescript": "^6.0.3", "vitest": "^4.1.6" }, diff --git a/packages/connector-mysql/package.json b/packages/connector-mysql/package.json index dc2df809..16bbc782 100644 --- a/packages/connector-mysql/package.json +++ b/packages/connector-mysql/package.json @@ -31,6 +31,7 @@ }, "devDependencies": { "@types/node": "^25.7.0", + "@vitest/coverage-v8": "^4.1.6", "typescript": "^6.0.3", "vitest": "^4.1.6" }, diff --git a/packages/connector-postgres/package.json b/packages/connector-postgres/package.json index 32991ec3..de7c5457 100644 --- a/packages/connector-postgres/package.json +++ b/packages/connector-postgres/package.json @@ -32,6 +32,7 @@ "devDependencies": { "@types/node": "^25.7.0", "@types/pg": "^8.20.0", + "@vitest/coverage-v8": "^4.1.6", "typescript": "^6.0.3", "vitest": "^4.1.6" }, diff --git a/packages/connector-snowflake/package.json b/packages/connector-snowflake/package.json index fb88c6a3..5135a457 100644 --- a/packages/connector-snowflake/package.json +++ b/packages/connector-snowflake/package.json @@ -31,6 +31,7 @@ }, "devDependencies": { "@types/node": "^25.7.0", + "@vitest/coverage-v8": "^4.1.6", "typescript": "^6.0.3", "vitest": "^4.1.6" }, diff --git a/packages/connector-sqlite/package.json b/packages/connector-sqlite/package.json index f83fdf0e..6dc0b725 100644 --- a/packages/connector-sqlite/package.json +++ b/packages/connector-sqlite/package.json @@ -32,6 +32,7 @@ "devDependencies": { "@types/better-sqlite3": "^7.6.13", "@types/node": "^25.7.0", + "@vitest/coverage-v8": "^4.1.6", "typescript": "^6.0.3", "vitest": "^4.1.6" }, diff --git a/packages/connector-sqlserver/package.json b/packages/connector-sqlserver/package.json index f737aeb3..cf8dac83 100644 --- a/packages/connector-sqlserver/package.json +++ b/packages/connector-sqlserver/package.json @@ -32,6 +32,7 @@ "devDependencies": { "@types/mssql": "^12.3.0", "@types/node": "^25.7.0", + "@vitest/coverage-v8": "^4.1.6", "typescript": "^6.0.3", "vitest": "^4.1.6" }, diff --git a/packages/context/package.json b/packages/context/package.json index 843df92d..32423c20 100644 --- a/packages/context/package.json +++ b/packages/context/package.json @@ -152,6 +152,7 @@ "@types/better-sqlite3": "^7.6.13", "@types/node": "^25.7.0", "@types/pg": "^8.20.0", + "@vitest/coverage-v8": "^4.1.6", "typescript": "^6.0.3", "vitest": "^4.1.6" }, diff --git a/packages/llm/package.json b/packages/llm/package.json index 57d89a43..277e0a98 100644 --- a/packages/llm/package.json +++ b/packages/llm/package.json @@ -34,6 +34,7 @@ }, "devDependencies": { "@types/node": "^25.7.0", + "@vitest/coverage-v8": "^4.1.6", "typescript": "^6.0.3", "vitest": "^4.1.6" }, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 6f9f19b1..de92ff9e 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -128,6 +128,9 @@ importers: '@types/react': specifier: ^19.2.14 version: 19.2.14 + '@vitest/coverage-v8': + specifier: ^4.1.6 + version: 4.1.6(vitest@4.1.6) better-sqlite3: specifier: ^12.10.0 version: 12.10.0 @@ -139,7 +142,7 @@ importers: version: 6.0.3 vitest: specifier: ^4.1.6 - version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) packages/connector-bigquery: dependencies: @@ -153,12 +156,15 @@ importers: '@types/node': specifier: ^24.3.0 version: 24.12.2 + '@vitest/coverage-v8': + specifier: ^4.1.6 + version: 4.1.6(vitest@4.1.6) typescript: specifier: ^6.0.3 version: 6.0.3 vitest: specifier: ^4.1.6 - version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) packages/connector-clickhouse: dependencies: @@ -172,12 +178,15 @@ importers: '@types/node': specifier: ^24.3.0 version: 24.12.2 + '@vitest/coverage-v8': + specifier: ^4.1.6 + version: 4.1.6(vitest@4.1.6) typescript: specifier: ^6.0.3 version: 6.0.3 vitest: specifier: ^4.1.6 - version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) packages/connector-mysql: dependencies: @@ -191,12 +200,15 @@ importers: '@types/node': specifier: ^24.3.0 version: 24.12.2 + '@vitest/coverage-v8': + specifier: ^4.1.6 + version: 4.1.6(vitest@4.1.6) typescript: specifier: ^6.0.3 version: 6.0.3 vitest: specifier: ^4.1.6 - version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) packages/connector-postgres: dependencies: @@ -213,12 +225,15 @@ importers: '@types/pg': specifier: ^8.20.0 version: 8.20.0 + '@vitest/coverage-v8': + specifier: ^4.1.6 + version: 4.1.6(vitest@4.1.6) typescript: specifier: ^6.0.3 version: 6.0.3 vitest: specifier: ^4.1.6 - version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) packages/connector-snowflake: dependencies: @@ -232,12 +247,15 @@ importers: '@types/node': specifier: ^24.3.0 version: 24.12.2 + '@vitest/coverage-v8': + specifier: ^4.1.6 + version: 4.1.6(vitest@4.1.6) typescript: specifier: ^6.0.3 version: 6.0.3 vitest: specifier: ^4.1.6 - version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) packages/connector-sqlite: dependencies: @@ -254,12 +272,15 @@ importers: '@types/node': specifier: ^24.3.0 version: 24.12.2 + '@vitest/coverage-v8': + specifier: ^4.1.6 + version: 4.1.6(vitest@4.1.6) typescript: specifier: ^6.0.3 version: 6.0.3 vitest: specifier: ^4.1.6 - version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) packages/connector-sqlserver: dependencies: @@ -276,12 +297,15 @@ importers: '@types/node': specifier: ^24.3.0 version: 24.12.2 + '@vitest/coverage-v8': + specifier: ^4.1.6 + version: 4.1.6(vitest@4.1.6) typescript: specifier: ^6.0.3 version: 6.0.3 vitest: specifier: ^4.1.6 - version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) packages/context: dependencies: @@ -349,12 +373,15 @@ importers: '@types/pg': specifier: ^8.20.0 version: 8.20.0 + '@vitest/coverage-v8': + specifier: ^4.1.6 + version: 4.1.6(vitest@4.1.6) typescript: specifier: ^6.0.3 version: 6.0.3 vitest: specifier: ^4.1.6 - version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) packages/llm: dependencies: @@ -377,12 +404,15 @@ importers: '@types/node': specifier: ^24.3.0 version: 24.12.2 + '@vitest/coverage-v8': + specifier: ^4.1.6 + version: 4.1.6(vitest@4.1.6) typescript: specifier: ^6.0.3 version: 6.0.3 vitest: specifier: ^4.1.6 - version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) packages: @@ -750,6 +780,27 @@ packages: resolution: {integrity: sha512-SriLPKezypIsiZ+TtlFfE46uuBIap2HeaQVS78e1P7rz5OSbq0rsd52WE1mC5f7vAeLiXqv7I7oRhL3WFZEw3Q==} engines: {node: '>=18.0.0'} + '@babel/helper-string-parser@7.27.1': + resolution: {integrity: sha512-qMlSxKbpRlAridDExk92nSobyDdpPijUq2DW6oDnUqd0iOGxmQjyqhMIihI9+zv4LPyZdRje2cavWPbCbWm3eA==} + engines: {node: '>=6.9.0'} + + '@babel/helper-validator-identifier@7.28.5': + resolution: {integrity: sha512-qSs4ifwzKJSV39ucNjsvc6WVHs6b7S03sOh2OcHF9UHfVPqWWALUsNUVzhSBiItjRZoLHx7nIarVjqKVusUZ1Q==} + engines: {node: '>=6.9.0'} + + '@babel/parser@7.29.3': + resolution: {integrity: sha512-b3ctpQwp+PROvU/cttc4OYl4MzfJUWy6FZg+PMXfzmt/+39iHVF0sDfqay8TQM3JA2EUOyKcFZt75jWriQijsA==} + engines: {node: '>=6.0.0'} + hasBin: true + + '@babel/types@7.29.0': + resolution: {integrity: sha512-LwdZHpScM4Qz8Xw2iKSzS+cfglZzJGvofQICy7W7v4caru4EaAmyUuO6BGrbyQ2mYV11W0U8j5mBhd14dd3B0A==} + engines: {node: '>=6.9.0'} + + '@bcoe/v8-coverage@1.0.2': + resolution: {integrity: sha512-6zABk/ECA/QYSCQ1NGiVwwbQerUCZ+TQbp64Q3AgmfNvurHH0j8TtXa1qbShXA6qqkpAj4V5W8pP6mLe1mcMqA==} + engines: {node: '>=18'} + '@biomejs/biome@2.4.15': resolution: {integrity: sha512-j5VH3a/h/HXTKBM50MDMxRCzkeLv9S2XJcW2WgnZT1+xyisi+0bISrXR82gCX+8S9lvK0skEvHJRN+3Ktr2hlw==} engines: {node: '>=14.21.3'} @@ -2517,6 +2568,15 @@ packages: resolution: {integrity: sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug==} engines: {node: '>= 20'} + '@vitest/coverage-v8@4.1.6': + resolution: {integrity: sha512-36l628fQ/9a/8ihy97eOtEnvWQEdqULQOJtcaxtoNq0G1w3Mxd4szSahOaMM9/NGyZ+hyKcMtIW/WIxq0XQViQ==} + peerDependencies: + '@vitest/browser': 4.1.6 + vitest: 4.1.6 + peerDependenciesMeta: + '@vitest/browser': + optional: true + '@vitest/expect@4.1.6': resolution: {integrity: sha512-7EHDquPthALSV0jhhjgEW8FXaviMx7rSqu8W6oqCoAuOhKov814P99QDV1pxMA3QPv21YudvJngIhjrNI4opLg==} @@ -2627,6 +2687,9 @@ packages: resolution: {integrity: sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==} engines: {node: '>=12'} + ast-v8-to-istanbul@1.0.0: + resolution: {integrity: sha512-1fSfIwuDICFA4LKkCzRPO7F0hzFf0B7+Xqrl27ynQaa+Rh0e1Es0v6kWHPott3lU10AyAr7oKHa65OppjLn3Rg==} + astring@1.9.0: resolution: {integrity: sha512-LElXdjswlqjWrPpJFg1Fx4wpkOCxj1TDHlSV4PlaRxHGWko024xICaa97ZkMfs6DRKlCguiAI+rbXv5GWwXIkg==} hasBin: true @@ -3364,6 +3427,10 @@ packages: engines: {node: '>=0.4.7'} hasBin: true + has-flag@4.0.0: + resolution: {integrity: sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==} + engines: {node: '>=8'} + has-symbols@1.1.0: resolution: {integrity: sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==} engines: {node: '>= 0.4'} @@ -3414,6 +3481,9 @@ packages: html-entities@2.6.0: resolution: {integrity: sha512-kig+rMn/QOVRvr7c86gQ8lWXq+Hkv6CbAH1hLu+RG338StTpE8Z0b44SDVaqVu7HGKf27frdmUYEs9hTUX/cLQ==} + html-escaper@2.0.2: + resolution: {integrity: sha512-H2iMtd0I4Mt5eYiapRdIDjp+XzelXQ0tFE4JS7YFwFevXXMmOp9myNrUvCg0D6ws8iqkRPBfKHgbwig1SmlLfg==} + html-void-elements@3.0.0: resolution: {integrity: sha512-bEqo66MRXsUGxWHV5IP0PUiAWwoEjba4VCzg0LjFJBpchPaTfyfCKTG6bc5F8ucKec3q5y6qOdGyYTSBEvhCrg==} @@ -3548,6 +3618,18 @@ packages: isexe@2.0.0: resolution: {integrity: sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==} + istanbul-lib-coverage@3.2.2: + resolution: {integrity: sha512-O8dpsF+r0WV/8MNRKfnmrtCWhuKjxrq2w+jpzBL5UZKTi2LeVWnWOmWRxFlesJONmc+wLAGvKQZEOanko0LFTg==} + engines: {node: '>=8'} + + istanbul-lib-report@3.0.1: + resolution: {integrity: sha512-GCfE1mtsHGOELCU8e/Z7YWzpmybrx/+dSTfLrvY8qRmaY6zXTKWn6WQIjaAFw069icm6GVMNkgu0NzI4iPZUNw==} + engines: {node: '>=10'} + + istanbul-reports@3.2.0: + resolution: {integrity: sha512-HGYWWS/ehqTV3xN10i23tkPkpH46MLCIMFNCaaKNavAXTF1RkqxawEPtnjnGZ6XKSInBKkiOA5BKS+aZiY3AvA==} + engines: {node: '>=8'} + jiti@2.7.0: resolution: {integrity: sha512-AC/7JofJvZGrrneWNaEnJeOLUx+JlGt7tNa0wZiRPT4MY1wmfKjt2+6O2p2uz2+skll8OZZmJMNqeke7kKbNgQ==} hasBin: true @@ -3558,6 +3640,9 @@ packages: js-md4@0.3.2: resolution: {integrity: sha512-/GDnfQYsltsjRswQhN9fhv3EMw2sCpUdrdxyWDOUK7eyD++r3gRhzgiQgc/x4MAv2i1iuQ4lxO5mvqM3vj4bwA==} + js-tokens@10.0.0: + resolution: {integrity: sha512-lM/UBzQmfJRo9ABXbPWemivdCW8V2G8FHaHdypQaIy523snUjog0W71ayWXTjiR+ixeMyVHN2XcpnTd/liPg/Q==} + js-yaml@4.1.1: resolution: {integrity: sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==} hasBin: true @@ -3718,6 +3803,13 @@ packages: magic-string@0.30.21: resolution: {integrity: sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==} + magicast@0.5.2: + resolution: {integrity: sha512-E3ZJh4J3S9KfwdjZhe2afj6R9lGIN5Pher1pF39UGrXRqq/VDaGVIGN13BjHd2u8B61hArAGOnso7nBOouW3TQ==} + + make-dir@4.0.0: + resolution: {integrity: sha512-hXdUTZYIVOt1Ex//jAQi+wTZZpUpwBj/0QsOzqegb3rGMMeJiSEu5xLHnYfBrRV4RH2+OCSOO95Is/7x1WJ4bw==} + engines: {node: '>=10'} + markdown-extensions@2.0.0: resolution: {integrity: sha512-o5vL7aDWatOTX8LzaS1WMoaoxIiLRQJuIKKe2wAw6IeULDHaqbiqiggmx+pKvZDb1Sj+pE46Sn1T7lCqfFtg1Q==} engines: {node: '>=16'} @@ -4574,6 +4666,10 @@ packages: babel-plugin-macros: optional: true + supports-color@7.2.0: + resolution: {integrity: sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==} + engines: {node: '>=8'} + tagged-tag@1.0.0: resolution: {integrity: sha512-yEFYrVhod+hdNyx7g5Bnkkb0G6si8HJurOoOEgC8B/O0uXLHlaey/65KRv6cuWBNhBgHKAROVpc7QyYqE5gFng==} engines: {node: '>=20'} @@ -5892,6 +5988,21 @@ snapshots: transitivePeerDependencies: - supports-color + '@babel/helper-string-parser@7.27.1': {} + + '@babel/helper-validator-identifier@7.28.5': {} + + '@babel/parser@7.29.3': + dependencies: + '@babel/types': 7.29.0 + + '@babel/types@7.29.0': + dependencies: + '@babel/helper-string-parser': 7.27.1 + '@babel/helper-validator-identifier': 7.28.5 + + '@bcoe/v8-coverage@1.0.2': {} + '@biomejs/biome@2.4.15': optionalDependencies: '@biomejs/cli-darwin-arm64': 2.4.15 @@ -7607,6 +7718,20 @@ snapshots: '@vercel/oidc@3.2.0': {} + '@vitest/coverage-v8@4.1.6(vitest@4.1.6)': + dependencies: + '@bcoe/v8-coverage': 1.0.2 + '@vitest/utils': 4.1.6 + ast-v8-to-istanbul: 1.0.0 + istanbul-lib-coverage: 3.2.2 + istanbul-lib-report: 3.0.1 + istanbul-reports: 3.2.0 + magicast: 0.5.2 + obug: 2.1.1 + std-env: 4.1.0 + tinyrainbow: 3.1.0 + vitest: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) + '@vitest/expect@4.1.6': dependencies: '@standard-schema/spec': 1.1.0 @@ -7720,6 +7845,12 @@ snapshots: assertion-error@2.0.1: {} + ast-v8-to-istanbul@1.0.0: + dependencies: + '@jridgewell/trace-mapping': 0.3.31 + estree-walker: 3.0.3 + js-tokens: 10.0.0 + astring@1.9.0: {} async@3.2.6: {} @@ -8458,6 +8589,8 @@ snapshots: optionalDependencies: uglify-js: 3.19.3 + has-flag@4.0.0: {} + has-symbols@1.1.0: {} has-tostringtag@1.0.2: @@ -8584,6 +8717,8 @@ snapshots: html-entities@2.6.0: {} + html-escaper@2.0.2: {} + html-void-elements@3.0.0: {} http-errors@2.0.1: @@ -8714,12 +8849,27 @@ snapshots: isexe@2.0.0: {} + istanbul-lib-coverage@3.2.2: {} + + istanbul-lib-report@3.0.1: + dependencies: + istanbul-lib-coverage: 3.2.2 + make-dir: 4.0.0 + supports-color: 7.2.0 + + istanbul-reports@3.2.0: + dependencies: + html-escaper: 2.0.2 + istanbul-lib-report: 3.0.1 + jiti@2.7.0: {} jose@6.2.2: {} js-md4@0.3.2: {} + js-tokens@10.0.0: {} + js-yaml@4.1.1: dependencies: argparse: 2.0.1 @@ -8875,6 +9025,16 @@ snapshots: dependencies: '@jridgewell/sourcemap-codec': 1.5.5 + magicast@0.5.2: + dependencies: + '@babel/parser': 7.29.3 + '@babel/types': 7.29.0 + source-map-js: 1.2.1 + + make-dir@4.0.0: + dependencies: + semver: 7.7.4 + markdown-extensions@2.0.0: {} markdown-table@3.0.4: {} @@ -10176,6 +10336,10 @@ snapshots: client-only: 0.0.1 react: 19.2.6 + supports-color@7.2.0: + dependencies: + has-flag: 4.0.0 + tagged-tag@1.0.0: {} tailwind-merge@3.6.0: {} @@ -10386,7 +10550,7 @@ snapshots: jiti: 2.7.0 yaml: 2.9.0 - vitest@4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)): + vitest@4.1.6(@opentelemetry/api@1.9.0)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.6)(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)): dependencies: '@vitest/expect': 4.1.6 '@vitest/mocker': 4.1.6(vite@8.0.10(@types/node@24.12.2)(esbuild@0.28.0)(jiti@2.7.0)(yaml@2.9.0)) @@ -10411,6 +10575,7 @@ snapshots: optionalDependencies: '@opentelemetry/api': 1.9.0 '@types/node': 24.12.2 + '@vitest/coverage-v8': 4.1.6(vitest@4.1.6) transitivePeerDependencies: - msw diff --git a/pyproject.toml b/pyproject.toml index e6422fb6..67d2acbb 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -15,6 +15,7 @@ Issues = "https://github.com/kaelio/ktx/issues" dev = [ "pre-commit>=4.6.0", "pytest>=9.0.2", + "pytest-cov>=7.1.0", "ruff>=0.8.4", ] diff --git a/scripts/package-artifacts.mjs b/scripts/package-artifacts.mjs index ee785c27..d6d86cd7 100644 --- a/scripts/package-artifacts.mjs +++ b/scripts/package-artifacts.mjs @@ -937,8 +937,9 @@ try { const doctor = await run('pnpm', ['exec', 'ktx', 'status', '--no-input']); assert.ok([0, 1].includes(doctor.code), 'ktx status setup exit code must be 0 or 1'); - requireStdout('ktx status setup', doctor, /KTX setup doctor/); - requireStdout('ktx status setup', doctor, /Node 22\\+/); + requireStdout('ktx status setup', doctor, /KTX status/); + requireStdout('ktx status setup', doctor, /No project here yet\\./); + requireStdout('ktx status setup', doctor, /Before you can run ktx setup/); assert.equal(doctor.stderr, '', 'ktx status setup wrote unexpected stderr'); } finally { await rm(root, { recursive: true, force: true }); diff --git a/uv.lock b/uv.lock index 5531c8e3..f3ffc767 100644 --- a/uv.lock +++ b/uv.lock @@ -548,6 +548,7 @@ source = { virtual = "." } dev = [ { name = "pre-commit" }, { name = "pytest" }, + { name = "pytest-cov" }, { name = "ruff" }, ] @@ -557,6 +558,7 @@ dev = [ dev = [ { name = "pre-commit", specifier = ">=4.6.0" }, { name = "pytest", specifier = ">=9.0.2" }, + { name = "pytest-cov", specifier = ">=7.1.0" }, { name = "ruff", specifier = ">=0.8.4" }, ]