apunkt/ktx
1
0
Fork 0
mirror of https://github.com/Kaelio/ktx.git synced 2026-06-22 08:38:08 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat(connectors): generalize readiness and constraint handling (#212)

Browse source 
* feat(connectors): add postgres maxConnections

* feat(connectors): add mysql maxConnections

* feat(connectors): add sqlserver maxConnections

* feat(connectors): rename snowflake pool config

* docs: document connector maxConnections

* feat(scan): add constraint discovery warning helper

* feat(scan): carry structural warnings through reports

* feat(postgres): soft-fail denied constraint discovery

* feat(mysql): soft-fail denied constraint discovery

* feat(sqlserver): soft-fail denied constraint discovery

* feat(bigquery): soft-fail denied primary key discovery

* feat(snowflake): report denied primary key discovery

* test(scan): verify constraint discovery warnings

* feat(historic-sql): use shared readiness probes

* docs: document query history readiness probes

* test(historic-sql): verify readiness probe registry

* test(ingest): account for live database warnings artifact

* Add skip option for agent setup
This commit is contained in:
Andrey Avtomonov 2026-05-24 19:30:06 +02:00 • committed by GitHub
parent cfd1749ab9
commit 78b8a0c025
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
42 changed files with 2763 additions and 554 deletions
Download patch file Download diff file Expand all files Collapse all files

10
docs-site/content/docs/cli-reference/ktx-setup.mdx
View file

@ -134,6 +134,16 @@ window flag applies to BigQuery and Snowflake; Postgres reads the current
Enabling query history makes deep ingest readiness matter for later
`ktx ingest` runs.
When query history is enabled for PostgreSQL, Snowflake, or BigQuery,
`ktx setup` runs a non-blocking readiness probe after the connection test
passes. A failed probe still writes setup changes, prints the warehouse-specific
grant or extension remediation, and leaves query-history ingest disabled until
you fix the prerequisite.
For BigQuery, the remediation tells you to grant `roles/bigquery.resourceViewer`
on the BigQuery project, or grant a custom role that contains
`bigquery.jobs.listAll`.
### Context Sources
In interactive setup, after you configure a database, choose

4
docs-site/content/docs/cli-reference/ktx-status.mdx
View file

@ -21,7 +21,7 @@ ktx status [options]
| `--json` | Print JSON output | `false` |
| `-v`, `--verbose` | Show every check, including passing ones | `false` |
| `--validate` | Only validate the `ktx.yaml` schema; skip readiness checks | `false` |
| `--fast` | Skip checks that require external communication (Postgres query-history probe, Claude Code auth probe) | `false` |
| `--fast` | Skip checks that require external communication (query-history readiness probes and Claude Code auth probe) | `false` |
| `--no-input` | Disable interactive terminal input | - |
## Examples
@ -39,7 +39,7 @@ ktx status --verbose
# Validate ktx.yaml without running readiness checks
ktx status --validate
# Skip slow probes (Postgres pg_stat_statements, Claude Code auth)
# Skip slow probes (query-history readiness, Claude Code auth)
ktx status --fast
# Check a project from another directory

15
docs-site/content/docs/configuration/ktx-yaml.mdx
View file

@ -157,11 +157,14 @@ connections:
dataset_ids: [analytics, mart]
```
For Snowflake connections, set `maxSessions` when deep ingest needs more or
fewer concurrent warehouse sessions. The default is `4`. This caps all
concurrent Snowflake SQL work for that connector instance, including schema
introspection, table sampling, relationship profiling, relationship
validation, and read-only SQL execution.
For Postgres, MySQL, SQL Server, and Snowflake connections, set
`maxConnections` when scan or ingest work needs to stay below the target's
connection cap. Postgres, MySQL, and SQL Server default to `10`; Snowflake
defaults to `4`. This caps all concurrent SQL work for that connector instance,
including schema introspection, table sampling, relationship profiling,
relationship validation, and read-only SQL execution. BigQuery and ClickHouse
do not expose `maxConnections` because their connectors don't use client-side
connection pools.
For Postgres, BigQuery, and Snowflake, `historicSql` and `context.queryHistory`
toggle query-history ingest. The shape is connector-specific; the setup wizard
@ -517,7 +520,7 @@ the manifest.
| `relationships.maxLlmTablesPerBatch` | `int > 0` | `40` | Max tables included in a single LLM relationship-proposal batch. |
| `relationships.maxCandidatesPerColumn` | `int > 0` | `25` | Max join partners considered per column. |
| `relationships.profileSampleRows` | `int > 0` | `10000` | Rows sampled per table when profiling values for relationship inference. |
| `relationships.profileConcurrency` | `int > 0` | `4` | Parallel relationship-profile queries against the database. For Snowflake, effective database concurrency is also bounded by the connection's `maxSessions`. |
| `relationships.profileConcurrency` | `int > 0` | `4` | Parallel relationship-profile queries against the database. For pooled connectors, effective database concurrency is also bounded by the connection's `maxConnections`. |
| `relationships.validationConcurrency` | `int > 0` | `4` | Parallel relationship validation queries against the database. |
| `relationships.validationBudget` | `all` \| `int ≥ 0` | runtime default | Cap on validation queries per scan. `all` means unlimited. |

11
docs-site/content/docs/integrations/agent-clients.mdx
View file

@ -9,7 +9,9 @@ admin surface for setup, ingest, status, daemon lifecycle, and debugging.
Run `ktx setup` and select your agent client targets, or configure manually
using the snippets below. Choose **Ask data questions with ktx MCP** for agent
clients. Choose **Ask data questions + manage ktx with CLI commands** only when
a developer or operator agent also needs pinned `ktx` admin commands.
a developer or operator agent also needs pinned `ktx` admin commands. Choose
**Skip agent setup for now** to leave agent integration incomplete and run
`ktx setup --agents` later.
## Install with setup
@ -43,14 +45,19 @@ ktx setup --agents --target codex --global
manifest lets status checks report agent readiness and lets future cleanup
remove only files **ktx** installed.
The interactive command asks two questions:
The interactive command asks what agents can do first:
```txt
◆ What should agents be allowed to do with this ktx project?
│ ○ Ask data questions with ktx MCP
│ ○ Ask data questions + manage ktx with CLI commands
│ ○ Skip agent setup for now
└
```
If you choose an install mode, it then asks which targets to install:
```txt
◆ Which agent targets should ktx install?
│ ◻ Claude Code
│ ◻ Claude Desktop