apunkt/ktx
1
0
Fork 0
mirror of https://github.com/Kaelio/ktx.git synced 2026-07-13 11:22:11 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat(scan): pool Snowflake sessions and batch enrichment for faster ingest (#206)

Browse source 
* feat(cli): add RSA key-pair auth option to Snowflake setup wizard

Extends the interactive Snowflake setup flow with an authentication-method
prompt (password vs RSA/JWT key-pair). The RSA branch collects a private-key
path (env/file/absolute) and an optional passphrase; the resulting connection
config records `authMethod: 'rsa'` with `privateKey` and `passphrase` instead
of `password`.

* feat(scan): pool Snowflake sessions

* fix(scan): reuse structural snapshots and cleanup connectors

* feat(scan): parallelize relationship profiling

* feat(scan): batch table description generation

* docs: document Snowflake ingest concurrency knobs

* fix(scan): close Snowflake ingest perf verification gaps

* fix(scan): keep batched description failure bounded
This commit is contained in:
Andrey Avtomonov 2026-05-23 02:39:45 +02:00 • committed by GitHub
parent 6e31687782
commit 4e654c43c1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
23 changed files with 1247 additions and 237 deletions
Download patch file Download diff file Expand all files Collapse all files

4
packages/cli/src/context/scan/local-enrichment-artifacts.test.ts
View file

@ -289,6 +289,7 @@ describe('writeLocalScanEnrichmentArtifacts', () => {
maxLlmTablesPerBatch: 12,
maxCandidatesPerColumn: 7,
profileSampleRows: 500,
profileConcurrency: 3,
validationConcurrency: 2,
},
});
@ -378,6 +379,7 @@ describe('writeLocalScanEnrichmentArtifacts', () => {
validationRequiredForManifest: true,
maxCandidatesPerColumn: 7,
profileSampleRows: 500,
profileConcurrency: 3,
validationConcurrency: 2,
},
profileWarnings: [],
@ -472,6 +474,7 @@ describe('writeLocalScanEnrichmentArtifacts', () => {
maxLlmTablesPerBatch: 40,
maxCandidatesPerColumn: 25,
profileSampleRows: 10000,
profileConcurrency: 4,
validationConcurrency: 4,
},
dryRun: false,
@ -741,6 +744,7 @@ describe('writeLocalScanEnrichmentArtifacts', () => {
maxLlmTablesPerBatch: 40,
maxCandidatesPerColumn: 25,
profileSampleRows: 10000,
profileConcurrency: 4,
validationConcurrency: 4,
},
dryRun: false,