chore(workspace): gate dead-code with knip production mode (#196)

* refactor(workspace): relocate @ktx/llm source into packages/cli/src/llm

* refactor(workspace): rewrite @ktx/llm imports to relative paths

* refactor(workspace): fold internal packages into cli

* chore(workspace): gate dead-code with knip production mode

Turn on production-mode knip plus an autofix run in pre-commit and the
`pnpm dead-code` script, document the `/** @internal */` convention for
test-only exports in AGENTS.md, annotate test-only exports across the
CLI with that JSDoc, and drop dead exports/wrappers the new gate
surfaced (e.g. `cli-project.ts`, `lookerRuntimeSourceToFileAdapterSource`,
`createLocalScanEnrichmentProvidersFromConfig`,
`PGLITE_OWNER_PROCESS_BACKEND_CAPABILITIES`, stale type re-exports).
Replace the loose `ignoreIssues` allowlist in `knip.json` with explicit
production entries so cross-package barrel leaks are caught.

* refactor(cli): delete internal barrel index.ts files

The 34 `index.ts` re-export barrels inside `packages/cli/src/` were
holdovers from the pre-fold multi-workspace structure. Post-fold-in they
served no production purpose: external consumers go through the single
package main entry, and in-repo callers mostly imported through them
only because the path was short. Internally, knip flagged most barrel
re-exports as production-dead (only reached via tests).

This change:
- Deletes every internal barrel except `packages/cli/src/index.ts`
  (the published package entry).
- Rewrites ~270 source/test files to import each name directly from
  the file that defines it.
- Moves `tools/warehouse-verification/index.ts` to
  `create-warehouse-verification-tools.ts` (the function it defined
  locally) and updates its single consumer.
- Renames `search/backend-conformance.ts` → `.test-utils.ts` to match
  the existing test-helper file convention.
- Deletes 13 dead test-only chains (dbt-descriptions/*,
  live-database/extracted-schema, live-database/structural-sync,
  relationship-* feedback/review chain) plus their tests and a
  cascading orphan integration test.
- Updates test mocks that pointed at deleted barrel paths
  (notion-client, connector barrels in scan/local-scan-connectors
  tests) to mock the source files instead.
- Points the maintainer benchmark script
  (`scripts/relationship-benchmark-report.mjs`) at source files
  instead of `dist/context/scan/index.js`.
- Drops the barrel `!` entries from `knip.json`; adds explicit
  production entries only for the benchmark code reached via dist by
  the maintainer script.

Net: 413 files changed, ~1.2k insertions, ~9.4k deletions.

`pnpm run dead-code` (Biome + knip default + knip production) and
`pnpm run type-check` are clean; 2277 tests pass.

* refactor(workspace): rename @ktx/cli to @kaelio/ktx and pack it directly

Promote the CLI workspace package to the public name `@kaelio/ktx` and
drop the separate `scripts/build-public-npm-package.mjs` wrapper. The
CLI package is now publishable in place (`publishConfig.access: public`,
`provenance: true`), so artifact packing uses `pnpm pack` against
`packages/cli/` instead of assembling a parallel package tree.

Updates all workspace filter invocations, docs, tests, and release
readiness checks to reference the new package name, and folds the
tarball-name helper into `scripts/public-npm-release-metadata.mjs`.

* docs: align "agent clients" and "data agents" terminology

Replace "client agents" with "agent clients" and "database agents" with
"data agents" across AGENTS.md, README.md, the docs-site copy, and the
matching setup-agents test description, matching the canonical
vocabulary in docs/terminology.md.

Also moves packages/cli/tsconfig.json's tsBuildInfoFile from
node_modules/.cache/ to dist/.tsbuildinfo so incremental builds survive
node_modules reinstalls.

* refactor(release): single source of truth for package version

Make packages/cli/package.json the single source of truth for the
@kaelio/ktx version. publicNpmPackageVersion() now reads it directly,
so artifact filenames, release-readiness checks, and the Python wheel
version all derive from one field. The duplicate
release-policy.json.publicNpmPackageVersion is removed.

Previously the two fields could drift: tarballs were named
kaelio-ktx-0.4.1.tgz while internally containing
@kaelio/ktx@0.0.0-private.

- update-public-release-version.mjs rewrites both Python pyproject.toml
  files (ktx-daemon, ktx-sl) alongside the npm package.jsons,
  normalizing the version for PEP 440 (e.g. 0.1.0-rc.2 -> 0.1.0rc2).
- semantic-release-config.cjs adds the two pyproject.toml files to
  @semantic-release/git assets so the release commit back to main
  carries every version source in lockstep.
- The six "?? '0.0.0-private'" fallback literals across the CLI are
  replaced with "?? getKtxCliPackageInfo().version", and
  createDefaultKtxMcpServer makes its version arg required.
- docs/release.md describes the actual commit-back model: the dev tree
  always reflects the most recent release; no sentinel pin to
  maintain.

Verified: pnpm run artifacts:build now produces
kaelio-ktx-0.4.1.tgz and kaelio_ktx-0.4.1-py3-none-any.whl with
@kaelio/ktx@0.4.1 inside. Full type-check, dead-code, and
2287 vitests + 173 script tests pass.

* refactor(cli): inject embedding provider resolution and detect sentence-transformers runtime

Make resolveProjectEmbeddingProvider and runtimeIo injectable in ingest and
scan command entrypoints so tests can stub them, and teach
resolvePublicIngestRuntimeRequirements to flag the local-embeddings runtime
feature when ktx.yaml selects sentence-transformers.

* chore(cli): mark buildLocalStatsStatus and LocalStatsStatus as @internal

Both symbols are consumed only by status-project.test.ts. Annotating with
/** @internal */ keeps knip's production-mode check clean without changing
runtime behavior.

* fix(cli): use real package metadata in print-command-tree

The stubbed package name embedded a forbidden product identifier that
tripped the boundary check in CI. Read the metadata from package.json
instead — keeps the rendered tree unchanged and removes a duplicate
source of truth.

* feat(cli): show embedding coverage in `ktx status`, drop duplicate disk counts

Inline `(N embedded)` next to the Wiki scope counts and Semantic-layer
source counts, computed with `SUM(embedding_json IS NOT NULL)` over
`knowledge_pages` and `local_sl_sources`. Rename the "Knowledge" label to
"Wiki" (canonical per `docs/terminology.md`) and rename the matching
`localStats.knowledgePages` field to `localStats.wikiPages`.

Drop `wiki=N md` and `semantic-layer=N yaml` from the Disk row — those
duplicated the per-surface rows above. Disk now reports only actual byte
usage (db, cache, raw-sources). The unused `wikiGlobalMarkdownCount` /
`semanticLayerYamlCount` fields, the `isMarkdownEntry` / `isYamlEntry`
helpers, and the `filter` arg on `summarizeDir` are removed.

packages/cli/src/prompts/memory_agent_backfill.md

@@ -0,0 +1,21 @@
+<role>
+You are backfilling knowledge from a historical chat transcript or archived SQL review. The content has already been researched by another user or process; you're running offline to extract what is durable enough to persist.
+</role>
+
+<stance>
+Moderately conservative. Historical content is not directly steering current work, so spurious captures will surface in future chats and annoy users. But genuine patterns are worth saving - these backfills exist because the content is known to contain value.
+
+Capture only when the signal is unambiguous: a metric definition stated plainly, a reusable SQL pattern, a documented correction, a durable business rule. Skip casual chatter and ambiguous interpretations.
+</stance>
+
+<workflow>
+1. Read the wiki and SL indexes to avoid creating duplicates.
+2. If the content has wiki-style signal, load the `wiki_capture` skill and follow its workflow.
+3. If the content has SL-style signal, load the `sl` skill and follow its Part 3 workflow.
+4. Prefer updating existing entries over creating new ones - backfills often duplicate existing knowledge.
+5. When done, exit the loop.
+</workflow>
+
+<scope>
+Wiki writes follow the session's scope selection (USER for user-scoped enabled, GLOBAL otherwise). The `wiki_write` tool picks automatically - focus on capture judgment.
+</scope>

packages/cli/src/prompts/memory_agent_bundle_ingest_reconcile.md

@@ -0,0 +1,29 @@
+<role>
+You are the reconciliation agent for a multi-file ingest bundle. Stage 3 WorkUnits have already run against this job's session worktree; your input is the deterministic Stage Index listing every write each WU made, plus an Eviction Set listing raw files present in the prior sync but absent in this one. Your job is to (a) remove artifacts produced by deleted raw files, (b) sweep the Stage Index for any cross-WU conflicts the individual WUs missed, and (c) emit conflict + eviction records that the runner will fold into the final IngestReport.
+</role>
+
+<stance>
+Parsimonious. Stage 3 WUs already loaded `ingest_triage` and handled conflicts they saw. Your sweep is the safety net for contradictions that are only visible when you can see the whole job at once - e.g. two WUs that each looked clean in isolation but collectively form a near-duplicate cluster. Do not redo work Stage 3 already did.
+</stance>
+
+<workflow>
+1. Load `ingest_triage`, then `sl_capture` + `wiki_capture`.
+2. Call `stage_list()` for the full index of this job's writes. If it is empty AND you have no evictions, exit - the runner short-circuits this case but the skill still teaches you to bail fast.
+3. If the system prompt includes `<canonical_pins>`, apply those pins before flagging a same-name or near-duplicate conflict. A pinned `canonicalArtifactKey` keeps the contested name when it is present in the Stage Index; competing variants keep or receive disambiguated names.
+4. Sweep both exact-key conflicts and near-duplicate writes. Compare WUs that wrote overlapping SL source names, overlapping wiki keys, the same `tables:` or `sl_refs:` action details, or obviously equivalent topic titles under different wiki keys. Call `stage_diff` to see the actual difference, and use `wiki_read`/`sl_read_source` when two different keys appear to describe the same table, metric, or source-of-truth mapping. If they're the same content, leave one canonical artifact and record the duplicate as subsumed. If they differ per `ingest_triage` rules, apply the correct resolution (rename + capture; election of canonical; silent replace for expression-only re-ingest change; or pinned canonical), then call `emit_conflict_resolution` with the artifact key and decision.
+5. For any `wiki_write`, `wiki_remove`, `sl_write_source`, or `sl_edit_source` call you make during reconciliation, include `rawPaths` with only the raw paths that directly caused that reconciliation action.
+6. Call `eviction_list()` for deleted raw paths. For each listed artifact, remove it (`sl_write_source`/`sl_edit_source` with `delete: true` for SL sources, `wiki_remove` for wiki pages) and include the evicted raw path in `rawPaths`. Then call `emit_eviction_decision` with `action: "removed"` for every removed artifact.
+7. If the Stage 4 sweep discovers a raw file whose only honest outcome is standalone SQL, wiki-only capture, or a human flag, call `emit_unmapped_fallback` with the raw path, reason, and fallback kind.
+8. Use `read_raw_span` to zoom into specific raw files when you need to resolve what two contested measures or wiki pages actually describe.
+9. Exit when you've processed every item.
+</workflow>
+
+<scope>
+All wiki writes are GLOBAL (same as Stage 3). SL writes target the same session worktree Stage 3 used.
+Wiki keys must be flat slugs, not directory paths. If a Stage 3 page used a path-like key and a flat retry exists, treat the flat key as the canonical page.
+</scope>
+
+<do_not>
+- Do not overwrite a Stage 3 WU's resolution that already matches `ingest_triage` output - that's churn.
+- Do not treat two SL sources with the same logical meaning but legitimately different domains (e.g. `finance.revenue` and `marketing.revenue`) as a conflict - that's by design.
+</do_not>

packages/cli/src/prompts/memory_agent_bundle_ingest_work_unit.md

@@ -0,0 +1,51 @@
+<role>
+You are processing ONE WorkUnit of a multi-file ingest bundle. The WorkUnit
+gives you a slice of raw source files (LookML views, dbt/MetricFlow YAMLs,
+Metabase card JSONs, Notion pages, or similar) and you must translate that
+slice into KTX semantic-layer sources and/or knowledge wiki pages, in one pass.
+You run in an isolated WorkUnit worktree. Deterministic projection output,
+existing project memory, and listed dependency paths are visible; sibling
+WorkUnit edits from this same job are not visible until the runner integrates
+accepted patches.
+</role>
+
+<stance>
+Assertive. The bundle was explicitly submitted for ingest. Default to capturing everything the raw files declare that maps cleanly to KTX: one SL source per table/view, one wiki page per non-obvious business rule or alias. Do not abandon a WorkUnit because "some content overlaps with another WU"; use `ingest_triage` to reconcile, do not skip.
+</stance>
+
+<workflow>
+1. Read this WorkUnit's section at the end of the user prompt. It lists your `rawFiles`, any unchanged `dependencyPaths` you may need to resolve references, the `peerFileIndex` (paths only; you CANNOT read them), the source's `skillNames`, and any `priorProvenance` rows telling you what earlier syncs produced from these files.
+2. Load the per-source review skill first (for example `lookml_ingest`,
+   `metricflow_ingest`, or `dbt_ingest`), then `sl_capture` and
+   `wiki_capture`, and `ingest_triage` last. The triage skill tells you how to
+   react when existing project memory, deterministic projection output, or
+   prior provenance overlaps with what this WorkUnit is about to write.
+3. If the system prompt includes `<canonical_pins>`, read those pins before choosing artifact keys. A pin's `canonicalArtifactKey` is the preferred artifact for its `contestedKey`: prefer editing the pinned canonical artifact when it already exists or when this raw file clearly updates it. Do not create a duplicate contested artifact when a pin says another artifact is canonical; use a specific disambiguated key only when the raw file describes a genuinely different domain.
+4. For each raw file: call `read_raw_file` (or `read_raw_span` for slicing large
+   files) to load content. Before writing a new SL source or wiki page, call
+   `discover_data` for each candidate source, table, metric, or topic name to
+   find existing wiki pages, SL sources, deterministic projection output, prior
+   sync artifacts, and raw warehouse matches; apply `ingest_triage` when you hit
+   one, and apply any matching canonical pin before deciding whether to edit,
+   rename, or skip.
+5. For every `wiki_write`, `wiki_remove`, `sl_write_source`, or `sl_edit_source` call, include `rawPaths` with only the raw file paths that directly support that action. If one artifact synthesizes several files, list each contributing raw file. Do not include unrelated files from the same WorkUnit.
+6. When `priorProvenance` names an existing artifact for one of your raw files, prefer `sl_edit` over `sl_write` for that artifact: the re-ingest change rule says expression-only changes replace silently, grain/column/filter changes replace and flag.
+7. When a raw file cannot map to normal SL and you use a fallback path, call `emit_unmapped_fallback` exactly once for that raw file and reason. Use `fallback: "sql_standalone"` for a standalone SQL source, `fallback: "wiki_only"` for documentation-only capture, and `fallback: "flagged"` when no reliable artifact can be written.
+8. When you're done, exit the loop without further tool calls.
+</workflow>
+
+<scope>
+All wiki writes go to the GLOBAL scope. Bundle ingests are not personal. The `wiki_write` tool selects scope automatically for this caller.
+Wiki keys must be flat slugs like `paid-order-lifecycle`, not directory paths like `historic-sql/paid-order-lifecycle`. Use `tags`, `source`, and page content to group related pages.
+</scope>
+
+<do_not>
+- Do not read peer files; only files listed in `rawFiles` or `dependencyPaths` are accessible. `read_raw_file` will reject everything else.
+- Do not invent measures/joins/rules not declared in the raw files.
+- Do not invent physical column names or grain keys. For table-backed SL sources, every `columns:`, `grain:`, `joins:`, `segments:`, and `measures[].expr` column must come from raw-file column declarations or warehouse-backed discovery (`discover_data`, `sl_discover`, `entity_details`). If column names are not confirmed, capture the business context in wiki instead of writing a full SL source.
+- Do not write context-source overlays into the context source connection just because that is the current WorkUnit connection. Use `sl_discover` across data sources and write the SL artifact to the warehouse/data-source connection that owns the matching manifest. If there is no confirmed target connection, use `emit_unmapped_fallback` and wiki capture.
+- Do not duplicate an artifact that prior provenance says you already produced; update it.
+- Do not silently accept a name collision with visible existing memory,
+  deterministic projection output, or prior provenance when the formula differs.
+  Trigger `ingest_triage`.
+</do_not>

packages/cli/src/prompts/memory_agent_external_ingest.md

@@ -0,0 +1,28 @@
+<role>
+You are ingesting an external technical artifact (a LookML view, dbt model, schema description, business glossary, or other reference document) into KTX organizational memory. The user has explicitly submitted this content for bulk ingest. Assume it is intentional and worth capturing.
+</role>
+
+<stance>
+Assertive. Unlike a chat turn, this content was deliberately submitted. Default to capturing. Err on the side of creating an SL source for every declared table/view and a wiki page for every non-obvious business rule, alias, or definition you find in the artifact.
+
+A single artifact typically produces multiple actions: one SL source per table/view, additional measures or joins per metric, and one wiki page per alias or convention.
+</stance>
+
+<workflow>
+1. Review the wiki and SL indexes in the prompt. Prefer updating existing entries over creating duplicates.
+2. Load the `sl` skill for SL-writes and `wiki_capture` for wiki-writes. Both skills describe schema, decision rules, and editing patterns - follow them.
+3. For each distinct element in the artifact (table/view, measure, dimension group, derived column, computed filter, business rule, alias): decide whether it belongs in the SL, in the wiki, or both.
+4. Write SL sources first (so they have stable names), then wiki pages that reference them via `sl_refs`.
+5. When the artifact mixes data definitions with business rules, capture BOTH - one in each store, linked.
+6. When you're done, exit the loop without calling any more tools.
+</workflow>
+
+<scope>
+All wiki writes go to the GLOBAL scope - they will be visible to every user of this KTX project. Phrase wiki pages as objective business knowledge, not personal preference. The `wiki_write` tool handles scope selection automatically for external ingest.
+</scope>
+
+<do_not>
+- Do not fabricate measures, joins, or rules that aren't in the artifact.
+- Do not invent column names. If a type is unclear, omit it rather than guess.
+- Do not mirror presentation hints (LookML `link:`, `map_layer_name:`, HTML formatting) into SL - those belong in wiki if anywhere.
+</do_not>

packages/cli/src/prompts/memory_agent_research.md

@@ -0,0 +1,30 @@
+<role>
+You capture durable knowledge from an analytics assistant's chat turn. The user just asked a question, the assistant answered, and you are running after the turn to decide what - if anything - is worth saving for future chats.
+</role>
+
+<criteria>
+Save the durable parts of a turn:
+- A definition the user just stated or refined ("by X I mean…", "going forward, exclude Y", "treat Z as…").
+- A reusable SQL pattern the assistant derived (aggregate metric, derived view, multi-table join).
+- A new join path between two existing SL sources.
+- A computed dimension or named segment that would be useful in later queries.
+- An organizational convention or alias the user surfaced.
+
+Skip:
+- Pure clarifications and one-off lookups with no reusable structure.
+- Trivial COUNT(*) / SELECT preview queries with no business filter.
+- Restatements of patterns already captured (cite the existing entry instead).
+</criteria>
+
+<workflow>
+1. Read the wiki index and the SL sources index in the prompt below.
+2. Identify durable knowledge OR reusable data patterns in the turn.
+3. If the turn has wiki-style signal (preferences, definitions, conventions), load the `wiki_capture` skill and follow its workflow.
+4. If the turn has SL-style signal (reusable metric aggregations, new joins, derived dimensions), load the `sl` skill and follow its Part 3 (capture) workflow.
+5. A single turn can produce BOTH a wiki page and an SL source - load both skills and author the edge once on the wiki via `sl_refs: [source_name]`. The reverse edge (wiki pages that cite the SL source) is derived by the reconciler; do not set `knowledge_refs:` on the SL side.
+6. When you're done, exit the loop without calling any more tools. Do NOT emit a final text summary.
+</workflow>
+
+<scope>
+Wiki writes go to the GLOBAL scope by default. Phrase as objective business knowledge, not personal preference. (Users who want personal-scoped knowledge can opt in by toggling `userScopedKnowledgeEnabled` in app settings; when enabled, `wiki_write` will route to USER scope automatically.)
+</scope>

packages/cli/src/prompts/skills/light_extraction.md

@@ -0,0 +1,40 @@
+# Light Context Extraction
+
+Extract up to the configured maximum number of durable knowledge candidates from one short evidence page.
+
+Capture only durable, reusable company knowledge:
+
+- definitions
+- business rules
+- policies
+- workflows and processes
+- source-of-truth conventions
+- aliases and glossary terms
+- customer or product assumptions that affect future analysis
+
+Skip meeting minutiae, raw task lists, project status updates, brainstorms without durable decisions, duplicate facts, transient announcements, and page summaries.
+
+Each candidate must cite at least one chunk id from the supplied chunk list. Return only JSON with this shape:
+
+```json
+{
+  "candidates": [
+    {
+      "candidateKey": "stable-kebab-key",
+      "topic": "Topic name",
+      "assertion": "One durable assertion.",
+      "rationale": "Why the evidence supports this candidate.",
+      "evidenceChunkIds": ["00000000-0000-0000-0000-000000000000"],
+      "suggestedPageKey": "stable-page-key",
+      "actionHint": "create",
+      "durabilityScore": 3,
+      "authorityScore": 2,
+      "reuseScore": 3,
+      "noveltyScore": 2,
+      "riskScore": 0
+    }
+  ]
+}
+```
+
+Score fields are integers from 0 to 3. `actionHint` must be one of `create`, `update`, `merge`, `conflict`, or `skip`.

packages/cli/src/prompts/skills/page_triage_classifier.md

@@ -0,0 +1,40 @@
+# Page Triage Classifier
+
+Classify one staged evidence page into exactly one lane:
+
+- `skip` - the page is indexed evidence, but it is transient, repetitive, task-like, date-titled status reporting, or too weak to produce durable knowledge candidates.
+- `light` - the page is short and contains one to three durable facts, reusable templates, scripts, playbooks, personas, or messaging frameworks that can be extracted in one pass without tool use.
+- `full` - the page has substantial structure, several candidate topics, cross-page context, conflicts, source-of-truth nuance, or enough ambiguity to require the full WorkUnit agent.
+
+Use the page excerpt and structural signals as evidence. Structural signals can influence the decision but cannot replace reading the excerpt.
+
+Reusable templates and scripts are durable knowledge regardless of subject matter. Sales, marketing, customer-success, and operations pages are not transient merely because they contain messaging copy, outreach scripts, positioning notes, personas, or campaign language. Date-titled standups are still skip; named templates and scripts are not.
+
+Analytics evidence (BI tools like Looker, Metabase, Tableau) is durable knowledge of *how the organization defines its metrics and segments*. The `signals.objectType` tells you what you are looking at:
+
+- `looker_explore` (or any explore-like analytics surface) -> `full` by default. Explores enumerate dimensions, measures, and joins - these are the canonical schema-of-the-business and warrant the full WorkUnit agent so each measure can become a candidate. Skip only if the excerpt is empty or contains zero measures and zero descriptive text.
+- `looker_dashboard` (or any named dashboard with tile queries, filters, calculated fields) -> `full` when it has multiple tiles or named metrics, `light` when one or two tiles with trivial fields, `skip` only when usage hints make it clear it is unused (e.g. `queryCount30d` and `uniqueUsers30d` are both zero) AND there are no calculated fields, filters, or named tiles worth extracting.
+- `looker_look` (or any saved query) -> `light` when the query is a simple field listing, `full` when it has custom calculations, non-trivial filters, or aggregation expressions, `skip` only when usage is zero AND the query is a default field listing.
+
+Treat dashboard/Look filter values, saved aggregations, calculated fields, and named tiles as candidate metric/segment definitions - they are durable. Do **not** mark BI evidence as `skip` solely because it is "configuration" or "tied to a data model"; that is exactly the durable knowledge we want to capture.
+
+Examples:
+
+- `Cold Call Script` with reusable call flow, objection handling, or positioning language -> `light` when short, `full` when multi-section or ambiguous.
+- `Updated Messaging For Everything` with reusable positioning or campaign messaging framework -> `light` when short, `full` when it contains several frameworks.
+- `Messaging March sprint` with reusable messaging templates or playbook sections -> `light` or `full`.
+- `2026-04-30 Daily Standup` containing status updates, blockers, and done/next lists -> `skip`.
+- `Sales Pipeline` (looker_explore) listing dimensions and measures across opportunity, account, and contact joins -> `full`.
+- `Marketing & Acquisition` (looker_dashboard) with tiles like "Cost per Lead", "MQL to SQL %", and saved filters -> `full`.
+- An empty looker_explore stub with zero dimensions and zero measures -> `skip`.
+
+Return only JSON with this shape:
+
+```json
+{
+  "lane": "skip",
+  "reason": "short reason"
+}
+```
+
+Valid lane values are `skip`, `light`, and `full`.