feat(ingest): adapter-owned finalization replaces post-processor escape hatch (#136)

* Refine adapter-owned ingest finalization design after adversarial review iteration 1

* Refine adapter-owned ingest finalization design after adversarial review iteration 2

* Refine adapter-owned ingest finalization design after adversarial review iteration 3

* Implement adapter-owned ingest finalization v1

Moves finalization from runner-owned post-processors into typed
SourceAdapter.finalize() contracts. Adds finalization report schema,
scope derivation, override replay context, and migrates historic-SQL
projection. Removes IngestBundlePostProcessorPort wiring and
HistoricSqlProjectionPostProcessor.

* feat(ingest): export finalization adapter contract types

* test(ingest): exercise historic sql finalization locally

* docs(plans): add adapter-owned finalization v1 closure plan

* fix(setup): unblock clean Linux installs and add enabled_tables allowlist

- Pin managed Python runtime to 3.13 via `uv venv --python 3.13` so installs
  don't pick the system 3.12 on Ubuntu 24.04 and fail at wheel install.
- Sanitize NO_PROXY/no_proxy for the daemon child process — drop IPv6 CIDR
  entries that httpx rejects with InvalidURL (OrbStack injects these by
  default).
- Add `enabled_tables` allowlist on warehouse connections (zod schema +
  live-database introspection filter) to scope ingest to specific tables.
- Add `getting-started/troubleshooting-linux` docs page covering the Python
  3.13 prerequisite, IPv6 proxy gotcha, and a minimal working recipe; link
  it from the quickstart troubleshooting table and the llms-docs map.
- Make docs-site origin overridable via `KTX_DOCS_ORIGIN` so local builds
  can serve under host.docker.internal.

* Move docs changes to specs repo

* fix(cli): keep managed runtime python version private

* Deduplicate enabled tables filtering
This commit is contained in:
Andrey Avtomonov 2026-05-20 14:17:10 +02:00 committed by GitHub
parent fb82993ce1
commit 4ec5903aa5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
29 changed files with 1423 additions and 314 deletions

View file

@ -1020,9 +1020,16 @@ describe('runKtxIngest', () => {
sourceKey: 'historic-sql',
body: {
workUnits: [],
postProcessor: {
finalization: {
sourceKey: 'historic-sql',
status: 'success',
commitSha: 'finalization-sha',
touchedPaths: ['semantic-layer/warehouse/_schema/public.yaml', 'wiki/global/historic-sql-orders.md'],
declaredTouchedSources: [{ connectionId: 'warehouse', sourceName: 'orders' }],
derivedTouchedSources: [{ connectionId: 'warehouse', sourceName: 'orders' }],
declaredChangedWikiPageKeys: ['historic-sql-orders'],
derivedChangedWikiPageKeys: ['historic-sql-orders'],
mismatches: [],
result: {
tableUsageMerged: 56,
staleTablesMarked: 1,
@ -1032,7 +1039,24 @@ describe('runKtxIngest', () => {
},
errors: [],
warnings: [],
touchedSources: [],
actions: [
...Array.from({ length: 57 }, (_, index) => ({
target: 'sl' as const,
type: 'updated' as const,
key: `orders-${index}`,
detail: 'Merged usage',
targetConnectionId: 'warehouse',
rawPaths: ['tables/public/orders.json'],
})),
...Array.from({ length: 35 }, (_, index) => ({
target: 'wiki' as const,
type: 'updated' as const,
key: `historic-sql-orders-${index}`,
detail: 'Projected pattern',
rawPaths: ['patterns/orders.json'],
})),
],
provenanceExclusions: [],
},
},
}),