mirror of
https://github.com/Kaelio/ktx.git
synced 2026-06-13 08:15:14 +02:00
refactor: enforce ktx naming and AGENTS.md compliance sweep (#289)
Align the tree with AGENTS.md/CLAUDE.md conventions: - Rewrite user-facing strings, docs, and tests to lowercase `ktx` (no bare uppercase `KTX` tokens remain outside literal identifiers). - Drop the legacy `historicSql` migration path and its now-unused helpers, per the no-backward-compat rule. - Remove `as unknown as` / `any` casts: narrow `BaseTool` generics to `z.ZodObject`, add a typed `createLookerClient`, and delete the dead `getParametersSchema`/`toAnthropicFormat` pre-AI-SDK helpers. - Use `InvalidArgumentError` for Commander parse failures. - Finish the adapter→connector prose conversion in the `ktx.yaml` docs while keeping the literal `adapters` config key.
This commit is contained in:
parent
005c5fc860
commit
00cdf2de90
237 changed files with 844 additions and 974 deletions
|
|
@ -1,6 +1,6 @@
|
||||||
# Contributing to KTX
|
# Contributing to ktx
|
||||||
|
|
||||||
Thanks for your interest in KTX. This page covers **how to contribute** and
|
Thanks for your interest in **ktx**. This page covers **how to contribute** and
|
||||||
the **contributor rewards program**. For development setup, repository
|
the **contributor rewards program**. For development setup, repository
|
||||||
layout, and verification commands, see the
|
layout, and verification commands, see the
|
||||||
[Contributing guide in the docs](https://docs.kaelio.com/ktx/docs/community/contributing).
|
[Contributing guide in the docs](https://docs.kaelio.com/ktx/docs/community/contributing).
|
||||||
|
|
@ -23,7 +23,7 @@ layout, and verification commands, see the
|
||||||
## Contributor rewards program
|
## Contributor rewards program
|
||||||
|
|
||||||
We send merch to contributors whose pull requests get merged. The goal is
|
We send merch to contributors whose pull requests get merged. The goal is
|
||||||
to thank the people building KTX with us, not to drive volume.
|
to thank the people building **ktx** with us, not to drive volume.
|
||||||
|
|
||||||
### How it works
|
### How it works
|
||||||
|
|
||||||
|
|
@ -76,7 +76,7 @@ See the [Community & Support](https://docs.kaelio.com/ktx/docs/community/support
|
||||||
page for the full guide. The short version:
|
page for the full guide. The short version:
|
||||||
|
|
||||||
- **Questions, "how do I...", setup help, sharing patterns**: join the
|
- **Questions, "how do I...", setup help, sharing patterns**: join the
|
||||||
[KTX Slack](https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ).
|
[**ktx** Slack](https://join.slack.com/t/ktxcommunity/shared_invite/zt-3y9b44m1x-LVyNNJD5nwaZHq4XS29LMQ).
|
||||||
- **Bugs**: use the [Bug report](.github/ISSUE_TEMPLATE/bug_report.yml)
|
- **Bugs**: use the [Bug report](.github/ISSUE_TEMPLATE/bug_report.yml)
|
||||||
template.
|
template.
|
||||||
- **Feature requests**: use the
|
- **Feature requests**: use the
|
||||||
|
|
@ -87,7 +87,7 @@ page for the full guide. The short version:
|
||||||
|
|
||||||
## Code of conduct
|
## Code of conduct
|
||||||
|
|
||||||
KTX follows the
|
**ktx** follows the
|
||||||
[Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/).
|
[Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/).
|
||||||
Be respectful, assume good intent, and keep discussion focused on the
|
Be respectful, assume good intent, and keep discussion focused on the
|
||||||
project. Report concerns to the maintainers in Slack or by email at
|
project. Report concerns to the maintainers in Slack or by email at
|
||||||
|
|
|
||||||
|
|
@ -2,20 +2,20 @@
|
||||||
|
|
||||||
## Reporting a vulnerability
|
## Reporting a vulnerability
|
||||||
|
|
||||||
If you believe you've found a security vulnerability in KTX, please report it
|
If you believe you've found a security vulnerability in **ktx**, please report it
|
||||||
**privately** through GitHub Security Advisories:
|
**privately** through GitHub Security Advisories:
|
||||||
|
|
||||||
[Report a vulnerability](https://github.com/Kaelio/ktx/security/advisories/new)
|
[Report a vulnerability](https://github.com/Kaelio/ktx/security/advisories/new)
|
||||||
|
|
||||||
If you cannot use GitHub Security Advisories, email `support@kaelio.com`
|
If you cannot use GitHub Security Advisories, email `support@kaelio.com`
|
||||||
instead. Please do **not** open a public issue, post in the KTX Slack, or
|
instead. Please do **not** open a public issue, post in the **ktx** Slack, or
|
||||||
share details elsewhere until we have published a fix.
|
share details elsewhere until we have published a fix.
|
||||||
|
|
||||||
When reporting, please include:
|
When reporting, please include:
|
||||||
|
|
||||||
- A description of the issue and its impact
|
- A description of the issue and its impact
|
||||||
- Steps to reproduce
|
- Steps to reproduce
|
||||||
- The KTX version affected
|
- The **ktx** version affected
|
||||||
|
|
||||||
## What to expect
|
## What to expect
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -46,7 +46,7 @@ read, how to think, and where to put the results.
|
||||||
</p>
|
</p>
|
||||||
<ul className="mt-3 space-y-2 text-sm leading-6 text-fd-foreground">
|
<ul className="mt-3 space-y-2 text-sm leading-6 text-fd-foreground">
|
||||||
<li><code className="text-[13px] font-semibold">llm</code> - provider, models, prompt cache</li>
|
<li><code className="text-[13px] font-semibold">llm</code> - provider, models, prompt cache</li>
|
||||||
<li><code className="text-[13px] font-semibold">ingest</code> - adapters, embeddings, work units</li>
|
<li><code className="text-[13px] font-semibold">ingest</code> - connectors, embeddings, work units</li>
|
||||||
<li><code className="text-[13px] font-semibold">scan</code> - enrichment, relationships</li>
|
<li><code className="text-[13px] font-semibold">scan</code> - enrichment, relationships</li>
|
||||||
<li><code className="text-[13px] font-semibold">agent</code> - research-agent feature flags</li>
|
<li><code className="text-[13px] font-semibold">agent</code> - research-agent feature flags</li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
@ -440,7 +440,7 @@ provider-specific model identifiers.
|
||||||
## `ingest`
|
## `ingest`
|
||||||
|
|
||||||
`ingest` controls how **ktx** builds context from your stack. It lists the
|
`ingest` controls how **ktx** builds context from your stack. It lists the
|
||||||
adapters to run, the embedding provider used when adapters embed documents,
|
connectors to run, the embedding provider used when connectors embed documents,
|
||||||
and the concurrency and failure policy for work units.
|
and the concurrency and failure policy for work units.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
|
|
@ -471,12 +471,12 @@ ingest:
|
||||||
jitter: true
|
jitter: true
|
||||||
```
|
```
|
||||||
|
|
||||||
### Adapters
|
### Connectors
|
||||||
|
|
||||||
`adapters` is a list of adapter IDs that should run. Each ID matches a
|
`adapters` is a list of connector IDs that should run. Each ID matches a
|
||||||
connector that **ktx** ships locally:
|
connector that **ktx** ships locally:
|
||||||
|
|
||||||
| Adapter ID | What it ingests |
|
| Connector ID | What it ingests |
|
||||||
|------------|-----------------|
|
|------------|-----------------|
|
||||||
| `live-database` | Live warehouse introspection (schemas, tables, columns, samples). |
|
| `live-database` | Live warehouse introspection (schemas, tables, columns, samples). |
|
||||||
| `historic-sql` | Query history from Postgres `pg_stat_statements`, BigQuery `INFORMATION_SCHEMA.JOBS`, or Snowflake query history. |
|
| `historic-sql` | Query history from Postgres `pg_stat_statements`, BigQuery `INFORMATION_SCHEMA.JOBS`, or Snowflake query history. |
|
||||||
|
|
@ -486,7 +486,7 @@ connector that **ktx** ships locally:
|
||||||
| `looker` | Looker dashboards and looks via the API. |
|
| `looker` | Looker dashboards and looks via the API. |
|
||||||
| `metabase` | Metabase cards, dashboards, and database mappings. |
|
| `metabase` | Metabase cards, dashboards, and database mappings. |
|
||||||
| `notion` | Notion pages and databases for wiki context. |
|
| `notion` | Notion pages and databases for wiki context. |
|
||||||
| `fake` | Test/demo adapter. Useful in fixtures. |
|
| `fake` | Test/demo connector. Useful in fixtures. |
|
||||||
|
|
||||||
### Embeddings
|
### Embeddings
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -307,12 +307,12 @@ connection is unreachable or misconfigured the build is blocked up front and
|
||||||
**ktx** names the failing connection by id and connector type:
|
**ktx** names the failing connection by id and connector type:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
KTX cannot build context: a required connection failed its live test.
|
ktx cannot build context: a required connection failed its live test.
|
||||||
|
|
||||||
Failed connections:
|
Failed connections:
|
||||||
warehouse (postgres)
|
warehouse (postgres)
|
||||||
|
|
||||||
Each connection must be reachable before KTX builds context.
|
Each connection must be reachable before ktx builds context.
|
||||||
Run `ktx connection test <id>` to see the error, fix the connection, then retry.
|
Run `ktx connection test <id>` to see the error, fix the connection, then retry.
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
# KTX release runbook
|
# ktx release runbook
|
||||||
|
|
||||||
This runbook covers the maintainer workflow for publishing `@kaelio/ktx` to
|
This runbook covers the maintainer workflow for publishing `@kaelio/ktx` to
|
||||||
npm through GitHub Actions. The workflow uses semantic-release to choose the
|
npm through GitHub Actions. The workflow uses semantic-release to choose the
|
||||||
|
|
@ -36,7 +36,7 @@ Before you publish, confirm these requirements:
|
||||||
publish the first stable version as `0.1.0`.
|
publish the first stable version as `0.1.0`.
|
||||||
|
|
||||||
semantic-release doesn't support choosing an arbitrary first `0.x` stable
|
semantic-release doesn't support choosing an arbitrary first `0.x` stable
|
||||||
release. If KTX has no stable tag yet and you need the first stable release to
|
release. If **ktx** has no stable tag yet and you need the first stable release to
|
||||||
be `0.1.0`, create and push the baseline tag once before running the live
|
be `0.1.0`, create and push the baseline tag once before running the live
|
||||||
stable workflow:
|
stable workflow:
|
||||||
|
|
||||||
|
|
@ -46,7 +46,7 @@ git tag v0.0.0 "${root_commit}"
|
||||||
git push origin v0.0.0
|
git push origin v0.0.0
|
||||||
```
|
```
|
||||||
|
|
||||||
KTX follows the same versioning schema as the main Kaelio release workflow:
|
**ktx** follows the same versioning schema as the main Kaelio release workflow:
|
||||||
breaking-change and `major` commit markers create a minor release, not an
|
breaking-change and `major` commit markers create a minor release, not an
|
||||||
automatic major release. A major version requires an intentional manual release
|
automatic major release. A major version requires an intentional manual release
|
||||||
path.
|
path.
|
||||||
|
|
|
||||||
|
|
@ -17,7 +17,7 @@ The copied project initializes its own Git repository on first use.
|
||||||
|
|
||||||
## orbit-relationship-verification
|
## orbit-relationship-verification
|
||||||
|
|
||||||
`orbit-relationship-verification/` is a checked-in KTX project used by
|
`orbit-relationship-verification/` is a checked-in **ktx** project used by
|
||||||
`pnpm run relationships:verify-orbit`. It points the `orbit` SQLite connection
|
`pnpm run relationships:verify-orbit`. It points the `orbit` SQLite connection
|
||||||
at the Orbit-style no-declared-constraint relationship fixture and verifies that
|
at the Orbit-style no-declared-constraint relationship fixture and verifies that
|
||||||
relationship enrichment writes nine accepted joins without requiring a local
|
relationship enrichment writes nine accepted joins without requiring a local
|
||||||
|
|
@ -27,7 +27,7 @@ warehouse credential.
|
||||||
|
|
||||||
`postgres-historic/` is a manual Docker-backed smoke for Postgres
|
`postgres-historic/` is a manual Docker-backed smoke for Postgres
|
||||||
query-history ingest via `pg_stat_statements`. It verifies setup, staged
|
query-history ingest via `pg_stat_statements`. It verifies setup, staged
|
||||||
query-history artifacts, KTX daemon batch SQL analysis, bounded pattern
|
query-history artifacts, **ktx** daemon batch SQL analysis, bounded pattern
|
||||||
WorkUnit shards, and no-WorkUnit idempotency for unchanged bucketed table
|
WorkUnit shards, and no-WorkUnit idempotency for unchanged bucketed table
|
||||||
inputs and pattern shards.
|
inputs and pattern shards.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
# local-warehouse fixture
|
# local-warehouse fixture
|
||||||
|
|
||||||
This directory is a contributor fixture for KTX CLI smoke tests. It uses the
|
This directory is a contributor fixture for **ktx** CLI smoke tests. It uses the
|
||||||
internal fake ingest adapter so tests can run without a live database or
|
internal fake ingest adapter so tests can run without a live database or
|
||||||
external service.
|
external service.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,11 +1,11 @@
|
||||||
# Orbit-style relationship discovery verification
|
# Orbit-style relationship discovery verification
|
||||||
|
|
||||||
This KTX project backs the default `relationships:verify-orbit` command. It uses
|
This **ktx** project backs the default `relationships:verify-orbit` command. It uses
|
||||||
the checked-in Orbit-style SQLite fixture from the relationship discovery
|
the checked-in Orbit-style SQLite fixture from the relationship discovery
|
||||||
benchmark corpus, with no declared primary keys or foreign keys in the database
|
benchmark corpus, with no declared primary keys or foreign keys in the database
|
||||||
schema.
|
schema.
|
||||||
|
|
||||||
Run from the KTX workspace root:
|
Run from the **ktx** workspace root:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pnpm run relationships:verify-orbit
|
pnpm run relationships:verify-orbit
|
||||||
|
|
|
||||||
|
|
@ -14,7 +14,7 @@ generated local project.
|
||||||
The managed Python runtime smoke requires `uv` on `PATH`, isolates
|
The managed Python runtime smoke requires `uv` on `PATH`, isolates
|
||||||
`KTX_RUNTIME_ROOT`, verifies `ktx admin runtime status`, runs `ktx sl query --yes` to
|
`KTX_RUNTIME_ROOT`, verifies `ktx admin runtime status`, runs `ktx sl query --yes` to
|
||||||
install the core runtime from the bundled wheel, checks `ktx admin runtime status`,
|
install the core runtime from the bundled wheel, checks `ktx admin runtime status`,
|
||||||
starts and reuses the KTX daemon, and stops it.
|
starts and reuses the **ktx** daemon, and stops it.
|
||||||
|
|
||||||
The artifact manifest contains the public `@kaelio/ktx` npm tarball and the
|
The artifact manifest contains the public `@kaelio/ktx` npm tarball and the
|
||||||
bundled `kaelio-ktx` runtime wheel. The smoke does not install standalone
|
bundled `kaelio-ktx` runtime wheel. The smoke does not install standalone
|
||||||
|
|
|
||||||
|
|
@ -17,19 +17,19 @@ unchanged bounded pattern shards do not schedule LLM work.
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
- Docker with Compose v2
|
- Docker with Compose v2
|
||||||
- Node and pnpm matching the KTX workspace
|
- Node and pnpm matching the **ktx** workspace
|
||||||
- `uv` on `PATH` so the KTX-managed Python runtime can install the bundled
|
- `uv` on `PATH` so the **ktx**-managed Python runtime can install the bundled
|
||||||
runtime wheel
|
runtime wheel
|
||||||
|
|
||||||
## Run
|
## Run
|
||||||
|
|
||||||
From the KTX repository root:
|
From the **ktx** repository root:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
examples/postgres-historic/scripts/smoke.sh
|
examples/postgres-historic/scripts/smoke.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
The smoke creates a temporary KTX project, isolates the managed Python runtime
|
The smoke creates a temporary **ktx** project, isolates the managed Python runtime
|
||||||
under the temporary project parent, starts Postgres on `127.0.0.1:55432`, and
|
under the temporary project parent, starts Postgres on `127.0.0.1:55432`, and
|
||||||
uses this connection URL:
|
uses this connection URL:
|
||||||
|
|
||||||
|
|
@ -41,7 +41,7 @@ Set `KTX_POSTGRES_HISTORIC_KEEP_DOCKER=1` to leave the container running after
|
||||||
the script exits.
|
the script exits.
|
||||||
|
|
||||||
The smoke validates the query-history raw snapshot path without requiring LLM
|
The smoke validates the query-history raw snapshot path without requiring LLM
|
||||||
credentials. It uses KTX's local stage-only ingest API after `ktx setup`, so the
|
credentials. It uses **ktx**'s local stage-only ingest API after `ktx setup`, so the
|
||||||
deterministic reader, batch SQL parser, stable artifact writer, and diff-based
|
deterministic reader, batch SQL parser, stable artifact writer, and diff-based
|
||||||
WorkUnit planning are checked independently from curation.
|
WorkUnit planning are checked independently from curation.
|
||||||
|
|
||||||
|
|
@ -124,6 +124,6 @@ table.
|
||||||
- Missing grants: confirm `GRANT pg_read_all_stats TO ktx_reader;`.
|
- Missing grants: confirm `GRANT pg_read_all_stats TO ktx_reader;`.
|
||||||
- Empty snapshot: rerun `scripts/generate-workload.sh base` and keep
|
- Empty snapshot: rerun `scripts/generate-workload.sh base` and keep
|
||||||
`--query-history-min-executions 2` for the smoke.
|
`--query-history-min-executions 2` for the smoke.
|
||||||
- SQL-analysis failures: run `pnpm run ktx -- dev runtime status` from the KTX
|
- SQL-analysis failures: run `pnpm run ktx -- dev runtime status` from the **ktx**
|
||||||
repository root and confirm `uv`, the bundled Python wheel, and the managed
|
repository root and confirm `uv`, the bundled Python wheel, and the managed
|
||||||
runtime all pass.
|
runtime all pass.
|
||||||
|
|
|
||||||
|
|
@ -24,7 +24,7 @@ export function registerAdminCommands(program: Command, context: KtxCliCommandCo
|
||||||
|
|
||||||
admin
|
admin
|
||||||
.command('init')
|
.command('init')
|
||||||
.description('Initialize a Git-backed KTX project directory for maintenance scripts')
|
.description('Initialize a Git-backed ktx project directory for maintenance scripts')
|
||||||
.argument('[directory]', 'Project directory')
|
.argument('[directory]', 'Project directory')
|
||||||
.option('--force', 'Rewrite ktx.yaml and scaffold files in an existing project', false)
|
.option('--force', 'Rewrite ktx.yaml and scaffold files in an existing project', false)
|
||||||
.action(
|
.action(
|
||||||
|
|
|
||||||
|
|
@ -21,7 +21,7 @@ export function formatClaudeCodePromptCachingWarning(fields: string[]): string |
|
||||||
if (fields.length === 0) {
|
if (fields.length === 0) {
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
return `claude-code ignores ${fields.join(', ')} because the Claude Agent SDK does not expose KTX prompt-cache TTL, tool, or history markers.`;
|
return `claude-code ignores ${fields.join(', ')} because the Claude Agent SDK does not expose ktx prompt-cache TTL, tool, or history markers.`;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function formatClaudeCodePromptCachingFix(): string {
|
export function formatClaudeCodePromptCachingFix(): string {
|
||||||
|
|
|
||||||
|
|
@ -252,8 +252,8 @@ export function resolveCommandProjectDirOverride(command: CommandWithGlobalOptio
|
||||||
function createBaseProgram(info: KtxCliPackageInfo, io: KtxCliIo): Command {
|
function createBaseProgram(info: KtxCliPackageInfo, io: KtxCliIo): Command {
|
||||||
return new Command()
|
return new Command()
|
||||||
.name('ktx')
|
.name('ktx')
|
||||||
.description('KTX data agent context layer CLI')
|
.description('ktx data agent context layer CLI')
|
||||||
.option('--project-dir <path>', 'KTX project directory (default: KTX_PROJECT_DIR, nearest ktx.yaml, or cwd)')
|
.option('--project-dir <path>', 'ktx project directory (default: KTX_PROJECT_DIR, nearest ktx.yaml, or cwd)')
|
||||||
.option('--debug', 'Enable diagnostic logging to stderr')
|
.option('--debug', 'Enable diagnostic logging to stderr')
|
||||||
.version(`${info.name} ${info.version}`, '-v, --version', 'Show CLI version')
|
.version(`${info.name} ${info.version}`, '-v, --version', 'Show CLI version')
|
||||||
.helpOption('-h, --help', 'Show this help text')
|
.helpOption('-h, --help', 'Show this help text')
|
||||||
|
|
@ -466,7 +466,7 @@ export function buildKtxProgram(options: BuildKtxProgramOptions): Command {
|
||||||
const attachProjectGroup = shouldAttachCommandProjectGroup(path, hasProject);
|
const attachProjectGroup = shouldAttachCommandProjectGroup(path, hasProject);
|
||||||
telemetry.beginCommandSpan({
|
telemetry.beginCommandSpan({
|
||||||
commandPath: path,
|
commandPath: path,
|
||||||
flagsPresent: collectCommandFlagsPresent(commandNode as unknown as CommandUnknownOpts),
|
flagsPresent: collectCommandFlagsPresent(actionCommand),
|
||||||
projectDir: attachProjectGroup ? projectDir : undefined,
|
projectDir: attachProjectGroup ? projectDir : undefined,
|
||||||
hasProject,
|
hasProject,
|
||||||
attachProjectGroup,
|
attachProjectGroup,
|
||||||
|
|
|
||||||
|
|
@ -61,7 +61,7 @@ export function packageInfoFromJson(packageJson: unknown): KtxCliPackageInfo {
|
||||||
typeof packageJson.name !== 'string' ||
|
typeof packageJson.name !== 'string' ||
|
||||||
typeof packageJson.version !== 'string'
|
typeof packageJson.version !== 'string'
|
||||||
) {
|
) {
|
||||||
throw new Error('Invalid KTX CLI package metadata');
|
throw new Error('Invalid ktx CLI package metadata');
|
||||||
}
|
}
|
||||||
|
|
||||||
return {
|
return {
|
||||||
|
|
@ -77,7 +77,7 @@ async function runInit(args: { projectDir: string; force: boolean }, io: KtxCliI
|
||||||
force: args.force,
|
force: args.force,
|
||||||
});
|
});
|
||||||
|
|
||||||
io.stdout.write(`Initialized KTX project at ${result.projectDir}\n`);
|
io.stdout.write(`Initialized ktx project at ${result.projectDir}\n`);
|
||||||
io.stdout.write(`Config: ${result.configPath}\n`);
|
io.stdout.write(`Config: ${result.configPath}\n`);
|
||||||
io.stdout.write(`Commit: ${result.commitHash ?? 'none'}\n`);
|
io.stdout.write(`Commit: ${result.commitHash ?? 'none'}\n`);
|
||||||
return 0;
|
return 0;
|
||||||
|
|
|
||||||
|
|
@ -37,7 +37,7 @@ export function registerConnectionCommands(program: Command, context: KtxCliComm
|
||||||
connection
|
connection
|
||||||
.command('test')
|
.command('test')
|
||||||
.description('Test one or all configured connections (default: all)')
|
.description('Test one or all configured connections (default: all)')
|
||||||
.argument('[connectionId]', 'KTX connection id to test (omit to test all)')
|
.argument('[connectionId]', 'ktx connection id to test (omit to test all)')
|
||||||
.option('--all', 'Test every configured connection and print a summary list')
|
.option('--all', 'Test every configured connection and print a summary list')
|
||||||
.action(async (connectionId: string | undefined, options: { all?: boolean }, command) => {
|
.action(async (connectionId: string | undefined, options: { all?: boolean }, command) => {
|
||||||
if (options.all === true && connectionId !== undefined) {
|
if (options.all === true && connectionId !== undefined) {
|
||||||
|
|
|
||||||
|
|
@ -25,16 +25,16 @@ export function registerIngestCommands(
|
||||||
): void {
|
): void {
|
||||||
const ingest = program
|
const ingest = program
|
||||||
.command('ingest')
|
.command('ingest')
|
||||||
.description('Build or inspect KTX context, or capture text into memory')
|
.description('Build or inspect ktx context, or capture text into memory')
|
||||||
.usage('[options] [connectionId]')
|
.usage('[options] [connectionId]')
|
||||||
.argument('[connectionId]', 'Configured connection id to ingest (omit to ingest all)')
|
.argument('[connectionId]', 'Configured connection id to ingest (omit to ingest all)')
|
||||||
.option('--all', 'Ingest all configured connections', false)
|
.option('--all', 'Ingest all configured connections', false)
|
||||||
.addOption(new Option('--query-history', 'Include database query-history usage patterns').conflicts('noQueryHistory'))
|
.addOption(new Option('--query-history', 'Include database query-history usage patterns').conflicts('noQueryHistory'))
|
||||||
.addOption(new Option('--no-query-history', 'Skip database query-history usage patterns'))
|
.addOption(new Option('--no-query-history', 'Skip database query-history usage patterns'))
|
||||||
.option('--query-history-window-days <days>', 'Query-history lookback window for this run', parsePositiveIntegerOption)
|
.option('--query-history-window-days <days>', 'Query-history lookback window for this run', parsePositiveIntegerOption)
|
||||||
.option('--text <content>', 'Capture inline text into KTX memory; repeatable', collectOption, [])
|
.option('--text <content>', 'Capture inline text into ktx memory; repeatable', collectOption, [])
|
||||||
.option('--file <path>', 'Capture a text file into KTX memory; use - for stdin; repeatable', collectOption, [])
|
.option('--file <path>', 'Capture a text file into ktx memory; use - for stdin; repeatable', collectOption, [])
|
||||||
.option('--connection-id <connectionId>', 'KTX connection id to tag captured text/file notes')
|
.option('--connection-id <connectionId>', 'ktx connection id to tag captured text/file notes')
|
||||||
.option('--user-id <id>', 'Memory user id for text/file capture attribution', 'local-cli')
|
.option('--user-id <id>', 'Memory user id for text/file capture attribution', 'local-cli')
|
||||||
.option('--fail-fast', 'Stop after the first failed text/file item', false)
|
.option('--fail-fast', 'Stop after the first failed text/file item', false)
|
||||||
.addOption(new Option('--plain', 'Print plain text output').conflicts(['json']))
|
.addOption(new Option('--plain', 'Print plain text output').conflicts(['json']))
|
||||||
|
|
|
||||||
|
|
@ -27,11 +27,11 @@ function binPath(): string {
|
||||||
|
|
||||||
function formatMcpStartResultMessage(input: { status: 'started' | 'already-running'; url: string }): string {
|
function formatMcpStartResultMessage(input: { status: 'started' | 'already-running'; url: string }): string {
|
||||||
return [
|
return [
|
||||||
input.status === 'started' ? `KTX MCP daemon started: ${input.url}` : `KTX MCP daemon already running: ${input.url}`,
|
input.status === 'started' ? `ktx MCP daemon started: ${input.url}` : `ktx MCP daemon already running: ${input.url}`,
|
||||||
'',
|
'',
|
||||||
'KTX is ready for configured agents.',
|
'ktx is ready for configured agents.',
|
||||||
'Open your agent for this KTX project and ask a data question, for example:',
|
'Open your agent for this ktx project and ask a data question, for example:',
|
||||||
' "Use KTX to show me the available tables and metrics."',
|
' "Use ktx to show me the available tables and metrics."',
|
||||||
'',
|
'',
|
||||||
].join('\n');
|
].join('\n');
|
||||||
}
|
}
|
||||||
|
|
@ -50,14 +50,14 @@ async function printMcpStatus(context: KtxCliCommandContext, projectDir: string)
|
||||||
export function registerMcpCommands(program: Command, context: KtxCliCommandContext): void {
|
export function registerMcpCommands(program: Command, context: KtxCliCommandContext): void {
|
||||||
const mcp = program
|
const mcp = program
|
||||||
.command('mcp')
|
.command('mcp')
|
||||||
.description('Manage the KTX MCP HTTP server (bare command: show status)')
|
.description('Manage the ktx MCP HTTP server (bare command: show status)')
|
||||||
.action(async (_options, command) => {
|
.action(async (_options, command) => {
|
||||||
await printMcpStatus(context, resolveCommandProjectDir(command));
|
await printMcpStatus(context, resolveCommandProjectDir(command));
|
||||||
});
|
});
|
||||||
|
|
||||||
mcp
|
mcp
|
||||||
.command('stdio')
|
.command('stdio')
|
||||||
.description('Run the KTX MCP server over stdio')
|
.description('Run the ktx MCP server over stdio')
|
||||||
.action(async (_options, command) => {
|
.action(async (_options, command) => {
|
||||||
await (context.deps.mcp?.runStdioServer ?? runKtxMcpStdioServer)({
|
await (context.deps.mcp?.runStdioServer ?? runKtxMcpStdioServer)({
|
||||||
projectDir: resolveCommandProjectDir(command),
|
projectDir: resolveCommandProjectDir(command),
|
||||||
|
|
@ -68,7 +68,7 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
|
||||||
|
|
||||||
mcp
|
mcp
|
||||||
.command('start')
|
.command('start')
|
||||||
.description('Start the KTX MCP HTTP server')
|
.description('Start the ktx MCP HTTP server')
|
||||||
.option('--host <host>', 'Host to bind', '127.0.0.1')
|
.option('--host <host>', 'Host to bind', '127.0.0.1')
|
||||||
.option('--port <n>', 'Port to bind', parsePositiveIntegerOption, 7878)
|
.option('--port <n>', 'Port to bind', parsePositiveIntegerOption, 7878)
|
||||||
.option('--token <token>', 'Bearer token required for non-loopback binding')
|
.option('--token <token>', 'Bearer token required for non-loopback binding')
|
||||||
|
|
@ -96,7 +96,7 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
|
||||||
allowedOrigins: options.allowedOrigin,
|
allowedOrigins: options.allowedOrigin,
|
||||||
io: context.io,
|
io: context.io,
|
||||||
});
|
});
|
||||||
context.io.stdout.write(`KTX MCP server listening at http://${options.host}:${options.port}/mcp\n`);
|
context.io.stdout.write(`ktx MCP server listening at http://${options.host}:${options.port}/mcp\n`);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const result = await (context.deps.mcp?.startDaemon ?? startKtxMcpDaemon)({
|
const result = await (context.deps.mcp?.startDaemon ?? startKtxMcpDaemon)({
|
||||||
|
|
@ -114,24 +114,24 @@ export function registerMcpCommands(program: Command, context: KtxCliCommandCont
|
||||||
|
|
||||||
mcp
|
mcp
|
||||||
.command('stop')
|
.command('stop')
|
||||||
.description('Stop the KTX MCP daemon')
|
.description('Stop the ktx MCP daemon')
|
||||||
.action(async (_options, command) => {
|
.action(async (_options, command) => {
|
||||||
const result = await (context.deps.mcp?.stopDaemon ?? stopKtxMcpDaemon)({
|
const result = await (context.deps.mcp?.stopDaemon ?? stopKtxMcpDaemon)({
|
||||||
projectDir: resolveCommandProjectDir(command),
|
projectDir: resolveCommandProjectDir(command),
|
||||||
});
|
});
|
||||||
context.io.stdout.write(result.status === 'stopped' ? 'KTX MCP daemon stopped.\n' : 'KTX MCP daemon is not running.\n');
|
context.io.stdout.write(result.status === 'stopped' ? 'ktx MCP daemon stopped.\n' : 'ktx MCP daemon is not running.\n');
|
||||||
});
|
});
|
||||||
|
|
||||||
mcp
|
mcp
|
||||||
.command('status')
|
.command('status')
|
||||||
.description('Show KTX MCP daemon status')
|
.description('Show ktx MCP daemon status')
|
||||||
.action(async (_options, command) => {
|
.action(async (_options, command) => {
|
||||||
await printMcpStatus(context, resolveCommandProjectDir(command));
|
await printMcpStatus(context, resolveCommandProjectDir(command));
|
||||||
});
|
});
|
||||||
|
|
||||||
mcp
|
mcp
|
||||||
.command('logs')
|
.command('logs')
|
||||||
.description('Print the KTX MCP daemon log')
|
.description('Print the ktx MCP daemon log')
|
||||||
.option('--follow', 'Follow log output', false)
|
.option('--follow', 'Follow log output', false)
|
||||||
.action(async (options, command) => {
|
.action(async (options, command) => {
|
||||||
const logPath = mcpDaemonLayout(resolveCommandProjectDir(command)).logPath;
|
const logPath = mcpDaemonLayout(resolveCommandProjectDir(command)).logPath;
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,7 @@ async function runRuntimeArgs(context: KtxCliCommandContext, args: KtxRuntimeArg
|
||||||
export function registerRuntimeCommands(program: Command, context: KtxCliCommandContext): void {
|
export function registerRuntimeCommands(program: Command, context: KtxCliCommandContext): void {
|
||||||
const runtime = program
|
const runtime = program
|
||||||
.command('runtime')
|
.command('runtime')
|
||||||
.description('Install, start, stop, and inspect the KTX-managed Python runtime')
|
.description('Install, start, stop, and inspect the ktx-managed Python runtime')
|
||||||
.showHelpAfterError();
|
.showHelpAfterError();
|
||||||
|
|
||||||
runtime
|
runtime
|
||||||
|
|
@ -38,7 +38,7 @@ export function registerRuntimeCommands(program: Command, context: KtxCliCommand
|
||||||
|
|
||||||
runtime
|
runtime
|
||||||
.command('start')
|
.command('start')
|
||||||
.description('Start the KTX daemon')
|
.description('Start the ktx daemon')
|
||||||
.addOption(createRuntimeFeatureOption())
|
.addOption(createRuntimeFeatureOption())
|
||||||
.option('--force', 'Restart even when a matching daemon is already running', false)
|
.option('--force', 'Restart even when a matching daemon is already running', false)
|
||||||
.action(async (options: { feature: RuntimeFeature; force?: boolean }, command: CommandWithGlobalOptions) => {
|
.action(async (options: { feature: RuntimeFeature; force?: boolean }, command: CommandWithGlobalOptions) => {
|
||||||
|
|
@ -53,8 +53,8 @@ export function registerRuntimeCommands(program: Command, context: KtxCliCommand
|
||||||
|
|
||||||
runtime
|
runtime
|
||||||
.command('stop')
|
.command('stop')
|
||||||
.description('Stop the KTX daemon')
|
.description('Stop the ktx daemon')
|
||||||
.option('--all', 'Stop all KTX daemon processes recorded or discoverable on this machine', false)
|
.option('--all', 'Stop all ktx daemon processes recorded or discoverable on this machine', false)
|
||||||
.action(async (options: { all?: boolean }, command: CommandWithGlobalOptions) => {
|
.action(async (options: { all?: boolean }, command: CommandWithGlobalOptions) => {
|
||||||
await runRuntimeArgs(context, {
|
await runRuntimeArgs(context, {
|
||||||
command: 'stop',
|
command: 'stop',
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,7 @@ async function runSetupArgs(
|
||||||
function positiveInteger(value: string): number {
|
function positiveInteger(value: string): number {
|
||||||
const parsed = Number.parseInt(value, 10);
|
const parsed = Number.parseInt(value, 10);
|
||||||
if (!Number.isInteger(parsed) || parsed <= 0) {
|
if (!Number.isInteger(parsed) || parsed <= 0) {
|
||||||
throw new Error(`Expected a positive integer, received ${value}`);
|
throw new InvalidArgumentError(`Expected a positive integer, received ${value}`);
|
||||||
}
|
}
|
||||||
return parsed;
|
return parsed;
|
||||||
}
|
}
|
||||||
|
|
@ -202,8 +202,8 @@ function shouldShowSetupEntryMenu(
|
||||||
export function registerSetupCommands(program: Command, context: KtxCliCommandContext): void {
|
export function registerSetupCommands(program: Command, context: KtxCliCommandContext): void {
|
||||||
const setup = program
|
const setup = program
|
||||||
.command('setup')
|
.command('setup')
|
||||||
.description('Set up or resume a local KTX project')
|
.description('Set up or resume a local ktx project')
|
||||||
.addOption(new Option('--project-dir <path>', 'KTX project directory').hideHelp())
|
.addOption(new Option('--project-dir <path>', 'ktx project directory').hideHelp())
|
||||||
.option('--agents', 'Install agent integration only', false)
|
.option('--agents', 'Install agent integration only', false)
|
||||||
.addOption(
|
.addOption(
|
||||||
new Option('--target <target>', 'Agent target').choices([
|
new Option('--target <target>', 'Agent target').choices([
|
||||||
|
|
@ -295,7 +295,7 @@ export function registerSetupCommands(program: Command, context: KtxCliCommandCo
|
||||||
.hideHelp(),
|
.hideHelp(),
|
||||||
)
|
)
|
||||||
.addOption(
|
.addOption(
|
||||||
new Option('--skip-databases', 'Leave database setup incomplete; KTX cannot work until a database is added')
|
new Option('--skip-databases', 'Leave database setup incomplete; ktx cannot work until a database is added')
|
||||||
.hideHelp()
|
.hideHelp()
|
||||||
.default(false),
|
.default(false),
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -44,7 +44,7 @@ export function registerSlCommands(program: Command, context: KtxCliCommandConte
|
||||||
.description('List, search, validate, or query local semantic-layer sources')
|
.description('List, search, validate, or query local semantic-layer sources')
|
||||||
.usage('[options] [query...]')
|
.usage('[options] [query...]')
|
||||||
.argument('[query...]', 'Search query; omit to list all sources')
|
.argument('[query...]', 'Search query; omit to list all sources')
|
||||||
.option('--connection-id <id>', 'KTX connection id')
|
.option('--connection-id <id>', 'ktx connection id')
|
||||||
.option('--limit <number>', 'Maximum search results (search mode only)', parsePositiveIntegerOption)
|
.option('--limit <number>', 'Maximum search results (search mode only)', parsePositiveIntegerOption)
|
||||||
.addOption(
|
.addOption(
|
||||||
new Option('--output <mode>', 'Output mode: pretty (default in TTY), plain (TSV), or json').choices([
|
new Option('--output <mode>', 'Output mode: pretty (default in TTY), plain (TSV), or json').choices([
|
||||||
|
|
|
||||||
|
|
@ -26,7 +26,7 @@ export function registerSqlCommands(program: Command, context: KtxCliCommandCont
|
||||||
.command('sql')
|
.command('sql')
|
||||||
.description('Execute parser-validated read-only SQL against a configured connection')
|
.description('Execute parser-validated read-only SQL against a configured connection')
|
||||||
.argument('<sql...>', 'SQL query to execute')
|
.argument('<sql...>', 'SQL query to execute')
|
||||||
.requiredOption('-c, --connection <id>', 'KTX connection id')
|
.requiredOption('-c, --connection <id>', 'ktx connection id')
|
||||||
.option('--max-rows <n>', 'Maximum rows to return', parseSqlMaxRowsOption, DEFAULT_MAX_ROWS)
|
.option('--max-rows <n>', 'Maximum rows to return', parseSqlMaxRowsOption, DEFAULT_MAX_ROWS)
|
||||||
.addOption(
|
.addOption(
|
||||||
new Option('--output <mode>', 'Output mode: pretty (default), plain (TSV), or json').choices([
|
new Option('--output <mode>', 'Output mode: pretty (default), plain (TSV), or json').choices([
|
||||||
|
|
|
||||||
|
|
@ -15,7 +15,7 @@ function inputMode(options: { input?: boolean }): { inputMode?: 'disabled' } {
|
||||||
export function registerStatusCommands(program: Command, context: KtxCliCommandContext): void {
|
export function registerStatusCommands(program: Command, context: KtxCliCommandContext): void {
|
||||||
program
|
program
|
||||||
.command('status')
|
.command('status')
|
||||||
.description('Check current KTX setup and project readiness')
|
.description('Check current ktx setup and project readiness')
|
||||||
.option('--json', 'Print JSON output', false)
|
.option('--json', 'Print JSON output', false)
|
||||||
.option('-v, --verbose', 'Show every check, including passing ones', false)
|
.option('-v, --verbose', 'Show every check, including passing ones', false)
|
||||||
.option('--validate', 'Only validate the ktx.yaml schema; skip readiness checks', false)
|
.option('--validate', 'Only validate the ktx.yaml schema; skip readiness checks', false)
|
||||||
|
|
|
||||||
|
|
@ -134,7 +134,7 @@ async function createDefaultLookerClient(
|
||||||
connectionId: string,
|
connectionId: string,
|
||||||
): Promise<LookerTestPort> {
|
): Promise<LookerTestPort> {
|
||||||
const factory = new DefaultLookerConnectionClientFactory(createLocalLookerCredentialResolver(project));
|
const factory = new DefaultLookerConnectionClientFactory(createLocalLookerCredentialResolver(project));
|
||||||
return (await factory.createClient(connectionId)) as unknown as LookerTestPort;
|
return factory.createLookerClient(connectionId);
|
||||||
}
|
}
|
||||||
|
|
||||||
async function testLookerConnection(
|
async function testLookerConnection(
|
||||||
|
|
|
||||||
|
|
@ -645,7 +645,7 @@ export class KtxClickHouseScanConnector implements KtxScanConnector {
|
||||||
|
|
||||||
private assertConnection(connectionId: string): void {
|
private assertConnection(connectionId: string): void {
|
||||||
if (connectionId !== this.connectionId) {
|
if (connectionId !== this.connectionId) {
|
||||||
throw new Error(`KTX ClickHouse connector ${this.id} cannot serve connection ${connectionId}`);
|
throw new Error(`ktx ClickHouse connector ${this.id} cannot serve connection ${connectionId}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -794,7 +794,7 @@ export class KtxMysqlScanConnector implements KtxScanConnector {
|
||||||
|
|
||||||
private assertConnection(connectionId: string): void {
|
private assertConnection(connectionId: string): void {
|
||||||
if (connectionId !== this.connectionId) {
|
if (connectionId !== this.connectionId) {
|
||||||
throw new Error(`KTX MySQL connector ${this.id} cannot serve connection ${connectionId}`);
|
throw new Error(`ktx MySQL connector ${this.id} cannot serve connection ${connectionId}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -107,7 +107,7 @@ export interface KtxSnowflakeScanConnectorOptions {
|
||||||
connectionId: string;
|
connectionId: string;
|
||||||
connection: KtxSnowflakeConnectionConfig | undefined;
|
connection: KtxSnowflakeConnectionConfig | undefined;
|
||||||
/**
|
/**
|
||||||
* KTX project directory. When provided, snowflake-sdk's logger is redirected to
|
* ktx project directory. When provided, snowflake-sdk's logger is redirected to
|
||||||
* `<projectDir>/.ktx/logs/snowflake.log` so its JSON output does not bleed into
|
* `<projectDir>/.ktx/logs/snowflake.log` so its JSON output does not bleed into
|
||||||
* the CLI's TTY. Tests that use a fake driverFactory can leave this undefined.
|
* the CLI's TTY. Tests that use a fake driverFactory can leave this undefined.
|
||||||
*/
|
*/
|
||||||
|
|
|
||||||
|
|
@ -355,7 +355,7 @@ export class KtxSqliteScanConnector implements KtxScanConnector {
|
||||||
|
|
||||||
private assertConnection(connectionId: string): void {
|
private assertConnection(connectionId: string): void {
|
||||||
if (connectionId !== this.connectionId) {
|
if (connectionId !== this.connectionId) {
|
||||||
throw new Error(`KTX SQLite connector ${this.id} cannot serve connection ${connectionId}`);
|
throw new Error(`ktx SQLite connector ${this.id} cannot serve connection ${connectionId}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -833,7 +833,7 @@ export class KtxSqlServerScanConnector implements KtxScanConnector {
|
||||||
|
|
||||||
private assertConnection(connectionId: string): void {
|
private assertConnection(connectionId: string): void {
|
||||||
if (connectionId !== this.connectionId) {
|
if (connectionId !== this.connectionId) {
|
||||||
throw new Error(`KTX SQL Server connector ${this.id} cannot serve connection ${connectionId}`);
|
throw new Error(`ktx SQL Server connector ${this.id} cannot serve connection ${connectionId}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -415,7 +415,7 @@ export function renderContextBuildView(
|
||||||
const hasActive = allTargets.some((t) => t.status === 'running' || t.status === 'queued');
|
const hasActive = allTargets.some((t) => t.status === 'running' || t.status === 'queued');
|
||||||
const allDone = totalCount > 0 && !hasActive;
|
const allDone = totalCount > 0 && !hasActive;
|
||||||
|
|
||||||
const headerParts = [options.title ?? 'Building KTX context'];
|
const headerParts = [options.title ?? 'Building ktx context'];
|
||||||
if (totalCount > 0) {
|
if (totalCount > 0) {
|
||||||
const progressParts: string[] = [`${doneCount}/${totalCount}`];
|
const progressParts: string[] = [`${doneCount}/${totalCount}`];
|
||||||
if (state.totalElapsedMs > 0) progressParts.push(formatDuration(state.totalElapsedMs));
|
if (state.totalElapsedMs > 0) progressParts.push(formatDuration(state.totalElapsedMs));
|
||||||
|
|
@ -738,7 +738,7 @@ function failedStepDetail(result: KtxPublicIngestTargetResult): string | null {
|
||||||
const INTERNAL_FAILURE_LINE_RE =
|
const INTERNAL_FAILURE_LINE_RE =
|
||||||
/^(Report|Run|Job|Status|Adapter|Connection|Sync|Mode|Dry run|Diff|Tasks|Work units|Failed tasks|Saved memory|Provenance rows):\s*/;
|
/^(Report|Run|Job|Status|Adapter|Connection|Sync|Mode|Dry run|Diff|Tasks|Work units|Failed tasks|Saved memory|Provenance rows):\s*/;
|
||||||
const ACTIONABLE_FAILURE_LINE_RE =
|
const ACTIONABLE_FAILURE_LINE_RE =
|
||||||
/^(Missing bundled Python runtime manifest|KTX Python runtime is required|KTX daemon HTTP|Error:|Failed\b|Could not\b|Cannot\b)/;
|
/^(Missing bundled Python runtime manifest|ktx Python runtime is required|ktx daemon HTTP|Error:|Failed\b|Could not\b|Cannot\b)/;
|
||||||
|
|
||||||
function trimErrorPrefix(line: string): string {
|
function trimErrorPrefix(line: string): string {
|
||||||
return line.replace(/^Error:\s*/, '');
|
return line.replace(/^Error:\s*/, '');
|
||||||
|
|
@ -749,7 +749,7 @@ function firstCapturedFailureLine(output: string | undefined): string | null {
|
||||||
.split(/\r?\n/)
|
.split(/\r?\n/)
|
||||||
.map((candidate) => candidate.trim())
|
.map((candidate) => candidate.trim())
|
||||||
.filter((candidate) => candidate.length > 0)
|
.filter((candidate) => candidate.length > 0)
|
||||||
.filter((candidate) => !candidate.startsWith('KTX scan completed'))
|
.filter((candidate) => !candidate.startsWith('ktx scan completed'))
|
||||||
.filter((candidate) => !INTERNAL_FAILURE_LINE_RE.test(candidate));
|
.filter((candidate) => !INTERNAL_FAILURE_LINE_RE.test(candidate));
|
||||||
const line = lines.find((candidate) => ACTIONABLE_FAILURE_LINE_RE.test(candidate)) ?? lines.at(-1) ?? null;
|
const line = lines.find((candidate) => ACTIONABLE_FAILURE_LINE_RE.test(candidate)) ?? lines.at(-1) ?? null;
|
||||||
return line ? trimErrorPrefix(line) : null;
|
return line ? trimErrorPrefix(line) : null;
|
||||||
|
|
@ -789,7 +789,7 @@ function failureTextForTarget(input: {
|
||||||
const code = networkErrorCode(input.error, input.capturedOutput);
|
const code = networkErrorCode(input.error, input.capturedOutput);
|
||||||
if (code && isLocalSqlAnalysisConnectionRefused({ capturedOutput: input.capturedOutput, fallback: input.fallback })) {
|
if (code && isLocalSqlAnalysisConnectionRefused({ capturedOutput: input.capturedOutput, fallback: input.fallback })) {
|
||||||
return [
|
return [
|
||||||
`KTX could not reach the local SQL analysis runtime while processing query history for ${input.target.connectionId}.`,
|
`ktx could not reach the local SQL analysis runtime while processing query history for ${input.target.connectionId}.`,
|
||||||
`Reason: ${NETWORK_ERROR_REASONS[code]} (${code}).`,
|
`Reason: ${NETWORK_ERROR_REASONS[code]} (${code}).`,
|
||||||
`Retry: ${retryCommand({
|
`Retry: ${retryCommand({
|
||||||
projectDir: input.projectDir,
|
projectDir: input.projectDir,
|
||||||
|
|
@ -803,7 +803,7 @@ function failureTextForTarget(input: {
|
||||||
if (code) {
|
if (code) {
|
||||||
const operation = input.target.operation === 'database-ingest' ? 'reading schema for' : 'ingesting';
|
const operation = input.target.operation === 'database-ingest' ? 'reading schema for' : 'ingesting';
|
||||||
return [
|
return [
|
||||||
`KTX lost its connection to ${friendlyDriverName(input.target.driver)} while ${operation} ${input.target.connectionId}.`,
|
`ktx lost its connection to ${friendlyDriverName(input.target.driver)} while ${operation} ${input.target.connectionId}.`,
|
||||||
`Reason: ${NETWORK_ERROR_REASONS[code]} (${code}).`,
|
`Reason: ${NETWORK_ERROR_REASONS[code]} (${code}).`,
|
||||||
`Retry: ${retryCommand({
|
`Retry: ${retryCommand({
|
||||||
projectDir: input.projectDir,
|
projectDir: input.projectDir,
|
||||||
|
|
|
||||||
|
|
@ -88,13 +88,18 @@ const defaultLogger: LookerClientLogger = {
|
||||||
|
|
||||||
class InlineLookerSettings extends NodeSettings {
|
class InlineLookerSettings extends NodeSettings {
|
||||||
constructor(private readonly params: LookerConnectionParams) {
|
constructor(private readonly params: LookerConnectionParams) {
|
||||||
super('', {
|
// @looker/sdk-rtl boundary: NodeSettings consumes a string-valued config
|
||||||
|
// section (read back via the readConfig override below), but its constructor
|
||||||
|
// is typed to accept a fully-realized IApiSettings. The string record is the
|
||||||
|
// shape the library actually reads, so narrow to IApiSection first.
|
||||||
|
const settings: IApiSection = {
|
||||||
base_url: normalizeBaseUrl(params.base_url),
|
base_url: normalizeBaseUrl(params.base_url),
|
||||||
client_id: params.client_id,
|
client_id: params.client_id,
|
||||||
client_secret: params.client_secret, // pragma: allowlist secret
|
client_secret: params.client_secret, // pragma: allowlist secret
|
||||||
verify_ssl: 'true',
|
verify_ssl: 'true',
|
||||||
timeout: '120',
|
timeout: '120',
|
||||||
} as unknown as IApiSettings);
|
};
|
||||||
|
super('', settings as IApiSection & IApiSettings);
|
||||||
}
|
}
|
||||||
|
|
||||||
override readConfig(_section?: string): IApiSection {
|
override readConfig(_section?: string): IApiSection {
|
||||||
|
|
|
||||||
|
|
@ -19,6 +19,16 @@ export class DefaultLookerConnectionClientFactory implements LookerConnectionCli
|
||||||
) {}
|
) {}
|
||||||
|
|
||||||
async createClient(lookerConnectionId: string): Promise<LookerRuntimeClient> {
|
async createClient(lookerConnectionId: string): Promise<LookerRuntimeClient> {
|
||||||
|
return this.createLookerClient(lookerConnectionId);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Like {@link createClient} but preserves the concrete {@link LookerClient}
|
||||||
|
* type, so callers that need methods outside the `LookerRuntimeClient`
|
||||||
|
* contract (e.g. `listLookerConnections`, `testConnection`) keep them without
|
||||||
|
* a cast.
|
||||||
|
*/
|
||||||
|
async createLookerClient(lookerConnectionId: string): Promise<LookerClient> {
|
||||||
const credentials = await this.resolver.resolve(lookerConnectionId);
|
const credentials = await this.resolver.resolve(lookerConnectionId);
|
||||||
return new LookerClient(credentials, this.deps);
|
return new LookerClient(credentials, this.deps);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -214,7 +214,7 @@ export function validateLookerMappings(args: {
|
||||||
if (!args.knownKtxConnectionIds.has(mapping.ktxConnectionId)) {
|
if (!args.knownKtxConnectionIds.has(mapping.ktxConnectionId)) {
|
||||||
errors.push({
|
errors.push({
|
||||||
key: mapping.lookerConnectionName,
|
key: mapping.lookerConnectionName,
|
||||||
reason: `KTX connection ${mapping.ktxConnectionId} does not exist`,
|
reason: `ktx connection ${mapping.ktxConnectionId} does not exist`,
|
||||||
});
|
});
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -81,7 +81,7 @@ class MetabaseApiError extends Error {
|
||||||
* Strip Metabase `[[ ... {{ var }} ... ]]` optional-clause blocks from native SQL.
|
* Strip Metabase `[[ ... {{ var }} ... ]]` optional-clause blocks from native SQL.
|
||||||
*
|
*
|
||||||
* The bracketed blocks are emitted only when the embedded `{{ var }}` is supplied at
|
* The bracketed blocks are emitted only when the embedded `{{ var }}` is supplied at
|
||||||
* Metabase query time. For KTX semantic-layer ingest there's no such runtime
|
* Metabase query time. For ktx semantic-layer ingest there's no such runtime
|
||||||
* parameter — chat-time filters are composed by the SL query planner — so the optional
|
* parameter — chat-time filters are composed by the SL query planner — so the optional
|
||||||
* block must be removed before the SQL becomes a permanent SL source. Substituting a
|
* block must be removed before the SQL becomes a permanent SL source. Substituting a
|
||||||
* dummy value (the alternative) bakes a placeholder filter into the source and silently
|
* dummy value (the alternative) bakes a placeholder filter into the source and silently
|
||||||
|
|
|
||||||
|
|
@ -34,7 +34,7 @@ export function metabaseRuntimeConfigFromLocalConnection(
|
||||||
}
|
}
|
||||||
if (hasNetworkProxy(connection)) {
|
if (hasNetworkProxy(connection)) {
|
||||||
throw new Error(
|
throw new Error(
|
||||||
`Standalone KTX does not support proxy-bearing Metabase connections yet. Use hosted Metabase ingest for "${connectionId}" until the KTX Metabase proxy support spec lands.`,
|
`Standalone ktx does not support proxy-bearing Metabase connections yet. Use hosted Metabase ingest for "${connectionId}" until the ktx Metabase proxy support spec lands.`,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -186,7 +186,7 @@ export function validateMetabaseMappings(args: {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
if (!args.knownKtxConnectionIds.has(connectionId)) {
|
if (!args.knownKtxConnectionIds.has(connectionId)) {
|
||||||
errors.push({ key, reason: `KTX connection ${connectionId} does not exist` });
|
errors.push({ key, reason: `ktx connection ${connectionId} does not exist` });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return errors.length === 0 ? { ok: true } : { ok: false, errors };
|
return errors.length === 0 ? { ok: true } : { ok: false, errors };
|
||||||
|
|
@ -207,7 +207,7 @@ export function validateMappingPhysicalMatch(
|
||||||
}
|
}
|
||||||
|
|
||||||
if (target.connection_type !== expectedType) {
|
if (target.connection_type !== expectedType) {
|
||||||
return `Metabase database engine '${engine}' does not match KTX connection type '${target.connection_type}'`;
|
return `Metabase database engine '${engine}' does not match ktx connection type '${target.connection_type}'`;
|
||||||
}
|
}
|
||||||
|
|
||||||
const metabaseDb = normalizeName(mapping.metabaseDbName);
|
const metabaseDb = normalizeName(mapping.metabaseDbName);
|
||||||
|
|
@ -215,7 +215,7 @@ export function validateMappingPhysicalMatch(
|
||||||
|
|
||||||
if (engine === 'snowflake' || engine === 'bigquery' || engine === 'bigquery-cloud-sdk') {
|
if (engine === 'snowflake' || engine === 'bigquery' || engine === 'bigquery-cloud-sdk') {
|
||||||
if (metabaseDb && targetDb && metabaseDb !== targetDb) {
|
if (metabaseDb && targetDb && metabaseDb !== targetDb) {
|
||||||
return `Metabase database '${mapping.metabaseDbName}' does not match KTX connection database '${displayValue(
|
return `Metabase database '${mapping.metabaseDbName}' does not match ktx connection database '${displayValue(
|
||||||
getTargetDatabase(target),
|
getTargetDatabase(target),
|
||||||
)}'`;
|
)}'`;
|
||||||
}
|
}
|
||||||
|
|
@ -227,12 +227,12 @@ export function validateMappingPhysicalMatch(
|
||||||
const targetHost = normalizeHost(target.host);
|
const targetHost = normalizeHost(target.host);
|
||||||
|
|
||||||
if (metabaseHost && targetHost && metabaseHost !== targetHost) {
|
if (metabaseHost && targetHost && metabaseHost !== targetHost) {
|
||||||
return `Metabase host '${mapping.metabaseHost}' does not match KTX connection host '${displayValue(
|
return `Metabase host '${mapping.metabaseHost}' does not match ktx connection host '${displayValue(
|
||||||
target.host,
|
target.host,
|
||||||
)}'`;
|
)}'`;
|
||||||
}
|
}
|
||||||
if (metabaseDb && targetDb && metabaseDb !== targetDb) {
|
if (metabaseDb && targetDb && metabaseDb !== targetDb) {
|
||||||
return `Metabase database '${mapping.metabaseDbName}' does not match KTX connection database '${displayValue(
|
return `Metabase database '${mapping.metabaseDbName}' does not match ktx connection database '${displayValue(
|
||||||
getTargetDatabase(target),
|
getTargetDatabase(target),
|
||||||
)}'`;
|
)}'`;
|
||||||
}
|
}
|
||||||
|
|
@ -274,7 +274,7 @@ export async function refreshMetabaseMapping(args: {
|
||||||
if (!target) {
|
if (!target) {
|
||||||
physicalMismatches.push({
|
physicalMismatches.push({
|
||||||
mappingId: String(mapping.id),
|
mappingId: String(mapping.id),
|
||||||
reason: `KTX connection ${mapping.ktxConnectionId} does not exist`,
|
reason: `ktx connection ${mapping.ktxConnectionId} does not exist`,
|
||||||
});
|
});
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -62,7 +62,7 @@ async function readOptionalFile(path: string): Promise<{ exists: boolean; conten
|
||||||
|
|
||||||
function buildGateRepairSystemPrompt(): string {
|
function buildGateRepairSystemPrompt(): string {
|
||||||
return `<role>
|
return `<role>
|
||||||
You repair one KTX isolated-diff artifact gate failure inside the integration worktree.
|
You repair one ktx isolated-diff artifact gate failure inside the integration worktree.
|
||||||
</role>
|
</role>
|
||||||
|
|
||||||
<rules>
|
<rules>
|
||||||
|
|
|
||||||
|
|
@ -65,7 +65,7 @@ async function readOptionalFile(path: string): Promise<{ exists: boolean; conten
|
||||||
|
|
||||||
function buildResolverSystemPrompt(): string {
|
function buildResolverSystemPrompt(): string {
|
||||||
return `<role>
|
return `<role>
|
||||||
You repair one failed KTX isolated-diff patch inside the integration worktree.
|
You repair one failed ktx isolated-diff patch inside the integration worktree.
|
||||||
</role>
|
</role>
|
||||||
|
|
||||||
<rules>
|
<rules>
|
||||||
|
|
|
||||||
|
|
@ -77,7 +77,7 @@ import type { SourceAdapter } from './types.js';
|
||||||
|
|
||||||
const promptsDir = fileURLToPath(new URL('../../prompts', import.meta.url));
|
const promptsDir = fileURLToPath(new URL('../../prompts', import.meta.url));
|
||||||
const skillsDir = fileURLToPath(new URL('../../skills', import.meta.url));
|
const skillsDir = fileURLToPath(new URL('../../skills', import.meta.url));
|
||||||
const LOCAL_AUTHOR = { name: 'KTX Local', email: 'local@ktx.local' };
|
const LOCAL_AUTHOR = { name: 'ktx Local', email: 'local@ktx.local' };
|
||||||
const LOCAL_SHAPE_WARNING = 'Local ingest validates semantic-layer YAML shape only.';
|
const LOCAL_SHAPE_WARNING = 'Local ingest validates semantic-layer YAML shape only.';
|
||||||
const INGEST_TRACE_LEVELS = new Set<IngestTraceLevel>(['error', 'info', 'debug', 'trace']);
|
const INGEST_TRACE_LEVELS = new Set<IngestTraceLevel>(['error', 'info', 'debug', 'trace']);
|
||||||
|
|
||||||
|
|
@ -232,8 +232,9 @@ class LocalSlPythonPort implements SlPythonPort {
|
||||||
}
|
}
|
||||||
|
|
||||||
class LocalShapeOnlySlValidator implements SlValidatorPort<SlValidationDeps> {
|
class LocalShapeOnlySlValidator implements SlValidatorPort<SlValidationDeps> {
|
||||||
private validateParsedSource(sourceName: string, parsed: Record<string, unknown>) {
|
private validateParsedSource(sourceName: string, parsed: unknown) {
|
||||||
const isOverlay = parsed.table == null && parsed.sql == null;
|
const fields = (parsed ?? {}) as { table?: unknown; sql?: unknown };
|
||||||
|
const isOverlay = fields.table == null && fields.sql == null;
|
||||||
const result = (isOverlay ? sourceOverlaySchema : sourceDefinitionSchema).safeParse(parsed);
|
const result = (isOverlay ? sourceOverlaySchema : sourceDefinitionSchema).safeParse(parsed);
|
||||||
return result.success
|
return result.success
|
||||||
? { errors: [], warnings: [LOCAL_SHAPE_WARNING] }
|
? { errors: [], warnings: [LOCAL_SHAPE_WARNING] }
|
||||||
|
|
@ -255,7 +256,7 @@ class LocalShapeOnlySlValidator implements SlValidatorPort<SlValidationDeps> {
|
||||||
const { sources, loadErrors } = await deps.semanticLayerService.loadAllSources(connectionId);
|
const { sources, loadErrors } = await deps.semanticLayerService.loadAllSources(connectionId);
|
||||||
const source = sources.find((candidate) => candidate.name === sourceName);
|
const source = sources.find((candidate) => candidate.name === sourceName);
|
||||||
if (source) {
|
if (source) {
|
||||||
return this.validateParsedSource(sourceName, source as unknown as Record<string, unknown>);
|
return this.validateParsedSource(sourceName, source);
|
||||||
}
|
}
|
||||||
const detail =
|
const detail =
|
||||||
loadErrors.length > 0
|
loadErrors.length > 0
|
||||||
|
|
@ -279,7 +280,7 @@ class LocalShapeOnlySlValidator implements SlValidatorPort<SlValidationDeps> {
|
||||||
}
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const parsed = YAML.parse(file.content) as unknown as Record<string, unknown>;
|
const parsed = YAML.parse(file.content) as Record<string, unknown>;
|
||||||
return this.validateParsedSource(sourceName, parsed);
|
return this.validateParsedSource(sourceName, parsed);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
return {
|
return {
|
||||||
|
|
|
||||||
|
|
@ -513,7 +513,7 @@ export function buildMemoryFlowViewModel(input: MemoryFlowReplayInput): MemoryFl
|
||||||
: `${input.connectionId}/${input.adapter}`;
|
: `${input.connectionId}/${input.adapter}`;
|
||||||
|
|
||||||
return {
|
return {
|
||||||
title: `KTX memory flow ${titleSources} ${input.status}`,
|
title: `ktx memory flow ${titleSources} ${input.status}`,
|
||||||
subtitle: `Run ${input.runId} Sync ${input.syncId}`,
|
subtitle: `Run ${input.runId} Sync ${input.syncId}`,
|
||||||
status: input.status,
|
status: input.status,
|
||||||
activeLine: activeLine(input),
|
activeLine: activeLine(input),
|
||||||
|
|
|
||||||
|
|
@ -70,10 +70,10 @@ export class DiscoverDataTool extends BaseTool<typeof discoverDataInputSchema> {
|
||||||
}
|
}
|
||||||
|
|
||||||
if (input.sourceName) {
|
if (input.sourceName) {
|
||||||
const sl = await this.deps.slDiscoverTool.call(
|
const sl = (await this.deps.slDiscoverTool.call(
|
||||||
{ sourceName: input.sourceName, connectionId: input.connectionId },
|
{ sourceName: input.sourceName, connectionId: input.connectionId },
|
||||||
context,
|
context,
|
||||||
);
|
)) as ToolOutput;
|
||||||
return { markdown: sl.markdown, structured: { wiki: null, sl: sl.structured, raw: null } };
|
return { markdown: sl.markdown, structured: { wiki: null, sl: sl.structured, raw: null } };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -85,17 +85,17 @@ export class DiscoverDataTool extends BaseTool<typeof discoverDataInputSchema> {
|
||||||
let raw: DiscoverDataStructured['raw'] = null;
|
let raw: DiscoverDataStructured['raw'] = null;
|
||||||
|
|
||||||
if (query) {
|
if (query) {
|
||||||
const wikiResult = await this.deps.wikiSearchTool.call({ query, limit }, context);
|
const wikiResult = (await this.deps.wikiSearchTool.call({ query, limit }, context)) as ToolOutput;
|
||||||
if (totalFound(wikiResult.structured) > 0) {
|
if (totalFound(wikiResult.structured) > 0) {
|
||||||
parts.push('## Wiki Pages', '> use `wiki_read(blockKey)` for full content', wikiResult.markdown, '');
|
parts.push('## Wiki Pages', '> use `wiki_read(blockKey)` for full content', wikiResult.markdown, '');
|
||||||
wiki = wikiResult.structured;
|
wiki = wikiResult.structured;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const slResult = await this.deps.slDiscoverTool.call(
|
const slResult = (await this.deps.slDiscoverTool.call(
|
||||||
{ query: query || undefined, connectionId: input.connectionId },
|
{ query: query || undefined, connectionId: input.connectionId },
|
||||||
context,
|
context,
|
||||||
);
|
)) as ToolOutput;
|
||||||
if (totalSources(slResult.structured) > 0) {
|
if (totalSources(slResult.structured) > 0) {
|
||||||
parts.push(
|
parts.push(
|
||||||
'## Semantic Layer Sources',
|
'## Semantic Layer Sources',
|
||||||
|
|
|
||||||
|
|
@ -239,7 +239,7 @@ export class AiSdkKtxLlmRuntime implements KtxLlmRuntimePort {
|
||||||
const result = await this.generateTextWithRateLimitRetry(modelProviderName(model), input.abortSignal, () => generateText(request));
|
const result = await this.generateTextWithRateLimitRetry(modelProviderName(model), input.abortSignal, () => generateText(request));
|
||||||
input.onMetrics?.({ totalMs: Date.now() - startedAt, usage: toLlmTokenUsage(result.totalUsage ?? result.usage) });
|
input.onMetrics?.({ totalMs: Date.now() - startedAt, usage: toLlmTokenUsage(result.totalUsage ?? result.usage) });
|
||||||
if (typeof result.text !== 'string') {
|
if (typeof result.text !== 'string') {
|
||||||
throw new Error('KTX LLM text generation returned no text');
|
throw new Error('ktx LLM text generation returned no text');
|
||||||
}
|
}
|
||||||
return result.text;
|
return result.text;
|
||||||
}
|
}
|
||||||
|
|
@ -271,12 +271,12 @@ export class AiSdkKtxLlmRuntime implements KtxLlmRuntimePort {
|
||||||
}),
|
}),
|
||||||
}
|
}
|
||||||
: {}),
|
: {}),
|
||||||
output: Output.object({ schema: input.schema as unknown as FlexibleSchema<TOutput> }),
|
output: Output.object({ schema: input.schema as FlexibleSchema<TOutput> }),
|
||||||
};
|
};
|
||||||
const result = await this.generateTextWithRateLimitRetry(modelProviderName(model), input.abortSignal, () => generateText(request));
|
const result = await this.generateTextWithRateLimitRetry(modelProviderName(model), input.abortSignal, () => generateText(request));
|
||||||
input.onMetrics?.({ totalMs: Date.now() - startedAt, usage: toLlmTokenUsage(result.totalUsage ?? result.usage) });
|
input.onMetrics?.({ totalMs: Date.now() - startedAt, usage: toLlmTokenUsage(result.totalUsage ?? result.usage) });
|
||||||
if (result.output == null) {
|
if (result.output == null) {
|
||||||
throw new Error('KTX LLM object generation returned no output');
|
throw new Error('ktx LLM object generation returned no output');
|
||||||
}
|
}
|
||||||
return result.output as TOutput;
|
return result.output as TOutput;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -193,7 +193,7 @@ function isClaudeRateLimitResult(result: SDKResultMessage, rejectedSignal: RateL
|
||||||
}
|
}
|
||||||
|
|
||||||
function claudeRateLimitSignal(message: SDKMessage): RateLimitSignal | null {
|
function claudeRateLimitSignal(message: SDKMessage): RateLimitSignal | null {
|
||||||
const record = message as unknown as Record<string, unknown>;
|
const record = message as Record<string, unknown>;
|
||||||
if (record.type === 'rate_limit_event') {
|
if (record.type === 'rate_limit_event') {
|
||||||
const info = record.rate_limit_info as Record<string, unknown> | undefined;
|
const info = record.rate_limit_info as Record<string, unknown> | undefined;
|
||||||
if (!info) return null;
|
if (!info) return null;
|
||||||
|
|
@ -253,7 +253,7 @@ function baseOptions(input: {
|
||||||
? { behavior: 'allow', toolUseID: options.toolUseID }
|
? { behavior: 'allow', toolUseID: options.toolUseID }
|
||||||
: {
|
: {
|
||||||
behavior: 'deny',
|
behavior: 'deny',
|
||||||
message: `KTX claude-code runtime only permits current KTX MCP tools; denied ${toolName}.`,
|
message: `ktx claude-code runtime only permits current ktx MCP tools; denied ${toolName}.`,
|
||||||
toolUseID: options.toolUseID,
|
toolUseID: options.toolUseID,
|
||||||
},
|
},
|
||||||
permissionMode: 'dontAsk',
|
permissionMode: 'dontAsk',
|
||||||
|
|
|
||||||
|
|
@ -198,7 +198,7 @@ export function resolveLocalKtxEmbeddingConfig(
|
||||||
batchSize: config.batchSize,
|
batchSize: config.batchSize,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
throw new Error(`Unsupported KTX embedding backend: ${String((config as { backend?: string }).backend)}`);
|
throw new Error(`Unsupported ktx embedding backend: ${String((config as { backend?: string }).backend)}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** @internal */
|
/** @internal */
|
||||||
|
|
|
||||||
|
|
@ -28,7 +28,7 @@ export function normalizeKtxRuntimeToolOutput(value: unknown): KtxRuntimeToolOut
|
||||||
|
|
||||||
function assertObjectSchema(name: string, schema: z.ZodType): asserts schema is z.ZodObject<z.ZodRawShape> {
|
function assertObjectSchema(name: string, schema: z.ZodType): asserts schema is z.ZodObject<z.ZodRawShape> {
|
||||||
if (!(schema instanceof z.ZodObject)) {
|
if (!(schema instanceof z.ZodObject)) {
|
||||||
throw new Error(`KTX runtime tool "${name}" must use z.object input schema for claude-code`);
|
throw new Error(`ktx runtime tool "${name}" must use z.object input schema for claude-code`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -75,7 +75,7 @@ export function createRuntimeToolDescriptorFromAiTool(name: string, aiSdkTool: T
|
||||||
inputSchema: aiSdkTool.inputSchema as KtxRuntimeToolDescriptor['inputSchema'],
|
inputSchema: aiSdkTool.inputSchema as KtxRuntimeToolDescriptor['inputSchema'],
|
||||||
execute: async (input) => {
|
execute: async (input) => {
|
||||||
if (typeof aiSdkTool.execute !== 'function') {
|
if (typeof aiSdkTool.execute !== 'function') {
|
||||||
throw new Error(`KTX runtime tool "${name}" has no execute function`);
|
throw new Error(`ktx runtime tool "${name}" has no execute function`);
|
||||||
}
|
}
|
||||||
return normalizeKtxRuntimeToolOutput(
|
return normalizeKtxRuntimeToolOutput(
|
||||||
await aiSdkTool.execute(input as never, { toolCallId: `runtime-${name}` } as never),
|
await aiSdkTool.execute(input as never, { toolCallId: `runtime-${name}` } as never),
|
||||||
|
|
|
||||||
|
|
@ -56,12 +56,12 @@ const toolAnnotations = {
|
||||||
|
|
||||||
const toolDescriptions = {
|
const toolDescriptions = {
|
||||||
connection_list:
|
connection_list:
|
||||||
'List configured read-only data connections available to this KTX project. Use this before connection-scoped tools when the project may have multiple warehouses.',
|
'List configured read-only data connections available to this ktx project. Use this before connection-scoped tools when the project may have multiple warehouses.',
|
||||||
discover_data:
|
discover_data:
|
||||||
'Search across KTX wiki pages, semantic-layer sources, measures, dimensions, raw tables, and columns. Example: discover_data({ query: "monthly orders by customer", connectionId: "warehouse", kinds: ["sl_source", "table"] }).',
|
'Search across ktx wiki pages, semantic-layer sources, measures, dimensions, raw tables, and columns. Example: discover_data({ query: "monthly orders by customer", connectionId: "warehouse", kinds: ["sl_source", "table"] }).',
|
||||||
wiki_search:
|
wiki_search:
|
||||||
'Search KTX wiki pages for reusable business context. Example: wiki_search({ query: "revenue recognition", limit: 5 }).',
|
'Search ktx wiki pages for reusable business context. Example: wiki_search({ query: "revenue recognition", limit: 5 }).',
|
||||||
wiki_read: 'Read a KTX wiki page by key returned from wiki_search. Example: wiki_read({ key: "global/revenue" }).',
|
wiki_read: 'Read a ktx wiki page by key returned from wiki_search. Example: wiki_read({ key: "global/revenue" }).',
|
||||||
entity_details:
|
entity_details:
|
||||||
'Read table and column metadata from the latest live-database scan snapshot. Example: entity_details({ connectionId: "warehouse", entities: [{ table: { catalog: null, db: "public", name: "orders" }, columns: ["id"] }] }).',
|
'Read table and column metadata from the latest live-database scan snapshot. Example: entity_details({ connectionId: "warehouse", entities: [{ table: { catalog: null, db: "public", name: "orders" }, columns: ["id"] }] }).',
|
||||||
dictionary_search:
|
dictionary_search:
|
||||||
|
|
@ -71,9 +71,9 @@ const toolDescriptions = {
|
||||||
sl_query:
|
sl_query:
|
||||||
'Execute a semantic-layer query and return headers, rows, and total row count, plus correctness notes (e.g. compile-only or fan-out) when relevant. The generated SQL and full query plan are omitted by default; request them with include: ["sql"] and/or include: ["plan"]. Example: sl_query({ connectionId: "warehouse", measures: ["orders.order_count"], dimensions: [{ field: "orders.created_at", granularity: "month" }], include: ["sql"] }).',
|
'Execute a semantic-layer query and return headers, rows, and total row count, plus correctness notes (e.g. compile-only or fan-out) when relevant. The generated SQL and full query plan are omitted by default; request them with include: ["sql"] and/or include: ["plan"]. Example: sl_query({ connectionId: "warehouse", measures: ["orders.order_count"], dimensions: [{ field: "orders.created_at", granularity: "month" }], include: ["sql"] }).',
|
||||||
sql_execution:
|
sql_execution:
|
||||||
'Execute one parser-validated read-only SQL query against a configured KTX connection. Example: sql_execution({ connectionId: "warehouse", sql: "select count(*) from public.orders", maxRows: 100 }).',
|
'Execute one parser-validated read-only SQL query against a configured ktx connection. Example: sql_execution({ connectionId: "warehouse", sql: "select count(*) from public.orders", maxRows: 100 }).',
|
||||||
memory_ingest:
|
memory_ingest:
|
||||||
'Ingest free-form markdown knowledge into durable KTX memory. Use this for business rules, metric definitions, schema gotchas, recurring findings, or explicit user requests to remember something. Example: memory_ingest({ connectionId: "warehouse", content: "ARR is reported in cents in this warehouse." }).',
|
'Ingest free-form markdown knowledge into durable ktx memory. Use this for business rules, metric definitions, schema gotchas, recurring findings, or explicit user requests to remember something. Example: memory_ingest({ connectionId: "warehouse", content: "ARR is reported in cents in this warehouse." }).',
|
||||||
memory_ingest_status:
|
memory_ingest_status:
|
||||||
'Read the current or final status for a memory ingest run. Example: memory_ingest_status({ runId: "memory-run-1" }).',
|
'Read the current or final status for a memory ingest run. Example: memory_ingest_status({ runId: "memory-run-1" }).',
|
||||||
} satisfies Record<string, string>;
|
} satisfies Record<string, string>;
|
||||||
|
|
@ -856,7 +856,7 @@ export function registerKtxContextTools(deps: RegisterKtxContextToolsDeps): void
|
||||||
const ingestInput: MemoryAgentInput = {
|
const ingestInput: MemoryAgentInput = {
|
||||||
userId: userContext.userId,
|
userId: userContext.userId,
|
||||||
chatId: `mcp-${randomUUID()}`,
|
chatId: `mcp-${randomUUID()}`,
|
||||||
userMessage: 'Ingest external knowledge into KTX memory.',
|
userMessage: 'Ingest external knowledge into ktx memory.',
|
||||||
assistantMessage: input.content,
|
assistantMessage: input.content,
|
||||||
connectionId: input.connectionId,
|
connectionId: input.connectionId,
|
||||||
sourceType: 'external_ingest',
|
sourceType: 'external_ingest',
|
||||||
|
|
|
||||||
|
|
@ -52,7 +52,7 @@ import type {
|
||||||
|
|
||||||
const promptsDir = fileURLToPath(new URL('../../prompts', import.meta.url));
|
const promptsDir = fileURLToPath(new URL('../../prompts', import.meta.url));
|
||||||
const skillsDir = fileURLToPath(new URL('../../skills', import.meta.url));
|
const skillsDir = fileURLToPath(new URL('../../skills', import.meta.url));
|
||||||
const LOCAL_AUTHOR = { name: 'KTX Local', email: 'local@ktx.local' };
|
const LOCAL_AUTHOR = { name: 'ktx Local', email: 'local@ktx.local' };
|
||||||
const LOCAL_SHAPE_WARNING = 'Local memory ingest validates semantic-layer YAML shape only.';
|
const LOCAL_SHAPE_WARNING = 'Local memory ingest validates semantic-layer YAML shape only.';
|
||||||
|
|
||||||
export interface CreateLocalProjectMemoryIngestOptions {
|
export interface CreateLocalProjectMemoryIngestOptions {
|
||||||
|
|
|
||||||
|
|
@ -62,7 +62,7 @@ const llmSchema = z
|
||||||
models: z
|
models: z
|
||||||
.partialRecord(z.enum(KTX_MODEL_ROLES), z.string().min(1))
|
.partialRecord(z.enum(KTX_MODEL_ROLES), z.string().min(1))
|
||||||
.default({})
|
.default({})
|
||||||
.describe('Per-role model overrides keyed by KTX model role (e.g. "default", "triage"). Values are provider-specific model identifiers.'),
|
.describe('Per-role model overrides keyed by ktx model role (e.g. "default", "triage"). Values are provider-specific model identifiers.'),
|
||||||
promptCaching: promptCachingSchema.optional().describe('Optional prompt-caching tunables.'),
|
promptCaching: promptCachingSchema.optional().describe('Optional prompt-caching tunables.'),
|
||||||
})
|
})
|
||||||
.describe('LLM provider, per-role model overrides, and prompt-caching tunables.');
|
.describe('LLM provider, per-role model overrides, and prompt-caching tunables.');
|
||||||
|
|
@ -243,14 +243,14 @@ const storageSchema = z
|
||||||
state: z
|
state: z
|
||||||
.enum(KTX_STORAGE_STATES)
|
.enum(KTX_STORAGE_STATES)
|
||||||
.default('sqlite')
|
.default('sqlite')
|
||||||
.describe('Backend for KTX state storage. "sqlite" uses .ktx/db.sqlite; "postgres" expects a configured Postgres connection.'),
|
.describe('Backend for ktx state storage. "sqlite" uses .ktx/db.sqlite; "postgres" expects a configured Postgres connection.'),
|
||||||
search: z
|
search: z
|
||||||
.enum(KTX_SEARCH_BACKENDS)
|
.enum(KTX_SEARCH_BACKENDS)
|
||||||
.default('sqlite-fts5')
|
.default('sqlite-fts5')
|
||||||
.describe('Backend for search indexes. "sqlite-fts5" uses SQLite FTS5; "postgres-hybrid" uses Postgres lexical + vector hybrid search.'),
|
.describe('Backend for search indexes. "sqlite-fts5" uses SQLite FTS5; "postgres-hybrid" uses Postgres lexical + vector hybrid search.'),
|
||||||
git: storageGitSchema.prefault({}).describe('Git-backed storage commit policy.'),
|
git: storageGitSchema.prefault({}).describe('Git-backed storage commit policy.'),
|
||||||
})
|
})
|
||||||
.describe('Storage backends and commit policy for KTX state and search indexes.');
|
.describe('Storage backends and commit policy for ktx state and search indexes.');
|
||||||
|
|
||||||
const connectionSchema = connectionConfigSchema;
|
const connectionSchema = connectionConfigSchema;
|
||||||
|
|
||||||
|
|
@ -282,13 +282,13 @@ const ktxProjectConfigSchema = z
|
||||||
.record(z.string(), connectionSchema)
|
.record(z.string(), connectionSchema)
|
||||||
.default({})
|
.default({})
|
||||||
.describe('Map of connection ID to connector configuration. Keys are user-chosen names referenced elsewhere in the config.'),
|
.describe('Map of connection ID to connector configuration. Keys are user-chosen names referenced elsewhere in the config.'),
|
||||||
storage: storageSchema.prefault({}).describe('Storage backends and commit policy for KTX state and search indexes.'),
|
storage: storageSchema.prefault({}).describe('Storage backends and commit policy for ktx state and search indexes.'),
|
||||||
llm: llmSchema.prefault({}).describe('LLM provider, per-role model overrides, and prompt-caching tunables.'),
|
llm: llmSchema.prefault({}).describe('LLM provider, per-role model overrides, and prompt-caching tunables.'),
|
||||||
ingest: ingestSchema.prefault({}).describe('Ingest pipeline configuration.'),
|
ingest: ingestSchema.prefault({}).describe('Ingest pipeline configuration.'),
|
||||||
agent: agentSchema.prefault({}).describe('Agent feature configuration.'),
|
agent: agentSchema.prefault({}).describe('Agent feature configuration.'),
|
||||||
scan: scanSchema.prefault({}).describe('Schema-scan configuration: enrichment and relationship discovery.'),
|
scan: scanSchema.prefault({}).describe('Schema-scan configuration: enrichment and relationship discovery.'),
|
||||||
})
|
})
|
||||||
.describe('Configuration schema for KTX project files (ktx.yaml).');
|
.describe('Configuration schema for ktx project files (ktx.yaml).');
|
||||||
|
|
||||||
export type KtxProjectConfig = z.infer<typeof ktxProjectConfigSchema>;
|
export type KtxProjectConfig = z.infer<typeof ktxProjectConfigSchema>;
|
||||||
export type KtxProjectLlmConfig = z.infer<typeof llmSchema>;
|
export type KtxProjectLlmConfig = z.infer<typeof llmSchema>;
|
||||||
|
|
|
||||||
|
|
@ -102,7 +102,7 @@ const lookerConnectionSchema = z
|
||||||
.min(1)
|
.min(1)
|
||||||
.optional()
|
.optional()
|
||||||
.describe('Reference to Looker OAuth client secret (e.g. env:LOOKER_CLIENT_SECRET).'),
|
.describe('Reference to Looker OAuth client secret (e.g. env:LOOKER_CLIENT_SECRET).'),
|
||||||
mappings: lookerMappingsSchema.optional().describe('Looker connection-name to KTX warehouse mappings.'),
|
mappings: lookerMappingsSchema.optional().describe('Looker connection-name to ktx warehouse mappings.'),
|
||||||
})
|
})
|
||||||
.describe('Looker context-source connection.');
|
.describe('Looker context-source connection.');
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -15,7 +15,7 @@ export const metabaseMappingsSchema = z
|
||||||
databaseMappings: z
|
databaseMappings: z
|
||||||
.record(z.string(), stringTargetSchema)
|
.record(z.string(), stringTargetSchema)
|
||||||
.default({})
|
.default({})
|
||||||
.describe('Map of Metabase database ID (positive integer string) to KTX connection ID. Use null to explicitly unmap.'),
|
.describe('Map of Metabase database ID (positive integer string) to ktx connection ID. Use null to explicitly unmap.'),
|
||||||
syncEnabled: z
|
syncEnabled: z
|
||||||
.record(z.string(), z.boolean())
|
.record(z.string(), z.boolean())
|
||||||
.default({})
|
.default({})
|
||||||
|
|
@ -38,7 +38,7 @@ export const lookerMappingsSchema = z
|
||||||
connectionMappings: z
|
connectionMappings: z
|
||||||
.record(z.string().min(1), stringTargetSchema)
|
.record(z.string().min(1), stringTargetSchema)
|
||||||
.default({})
|
.default({})
|
||||||
.describe('Map of Looker connection name to KTX connection ID. Use null to explicitly unmap.'),
|
.describe('Map of Looker connection name to ktx connection ID. Use null to explicitly unmap.'),
|
||||||
})
|
})
|
||||||
.describe('Looker connection-to-warehouse mapping configuration.');
|
.describe('Looker connection-to-warehouse mapping configuration.');
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -123,7 +123,7 @@ export async function initKtxProject(options: InitKtxProjectOptions): Promise<In
|
||||||
|
|
||||||
const commit = await runtime.git.commitFiles(
|
const commit = await runtime.git.commitFiles(
|
||||||
['ktx.yaml', ...TRACKED_SCAFFOLD_FILES.map((file) => file.path)],
|
['ktx.yaml', ...TRACKED_SCAFFOLD_FILES.map((file) => file.path)],
|
||||||
`Initialize KTX project: ${projectName}`,
|
`Initialize ktx project: ${projectName}`,
|
||||||
authorName,
|
authorName,
|
||||||
authorEmail,
|
authorEmail,
|
||||||
);
|
);
|
||||||
|
|
|
||||||
|
|
@ -569,7 +569,7 @@ export class KtxDescriptionGenerator {
|
||||||
|
|
||||||
if (!connector.sampleTable) {
|
if (!connector.sampleTable) {
|
||||||
fallbackReason = 'capability_missing';
|
fallbackReason = 'capability_missing';
|
||||||
this.logger?.warn('KTX scan connector does not support table sampling; falling back to metadata-only prompt', {
|
this.logger?.warn('ktx scan connector does not support table sampling; falling back to metadata-only prompt', {
|
||||||
connectorId: input.connector.id,
|
connectorId: input.connector.id,
|
||||||
table: input.table.name,
|
table: input.table.name,
|
||||||
});
|
});
|
||||||
|
|
@ -690,7 +690,7 @@ export class KtxDescriptionGenerator {
|
||||||
let fallbackReason: 'capability_missing' | 'sampling_failed' | 'empty_sample' | null = null;
|
let fallbackReason: 'capability_missing' | 'sampling_failed' | 'empty_sample' | null = null;
|
||||||
if (!input.connector.sampleTable) {
|
if (!input.connector.sampleTable) {
|
||||||
fallbackReason = 'capability_missing';
|
fallbackReason = 'capability_missing';
|
||||||
this.logger?.warn('KTX scan connector does not support table sampling; falling back to metadata-only prompt', {
|
this.logger?.warn('ktx scan connector does not support table sampling; falling back to metadata-only prompt', {
|
||||||
connectorId: input.connector.id,
|
connectorId: input.connector.id,
|
||||||
table: input.table.name,
|
table: input.table.name,
|
||||||
});
|
});
|
||||||
|
|
@ -846,7 +846,7 @@ export class KtxDescriptionGenerator {
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!input.connector.sampleTable) {
|
if (!input.connector.sampleTable) {
|
||||||
this.logger?.warn('KTX scan connector does not support table sampling for data-source description generation', {
|
this.logger?.warn('ktx scan connector does not support table sampling for data-source description generation', {
|
||||||
connectorId: input.connector.id,
|
connectorId: input.connector.id,
|
||||||
});
|
});
|
||||||
return 'No accessible tables found in database';
|
return 'No accessible tables found in database';
|
||||||
|
|
@ -927,7 +927,7 @@ export class KtxDescriptionGenerator {
|
||||||
let columnValues = column.sampleValues;
|
let columnValues = column.sampleValues;
|
||||||
if (!columnValues || columnValues.length === 0) {
|
if (!columnValues || columnValues.length === 0) {
|
||||||
if (!input.connector.sampleColumn) {
|
if (!input.connector.sampleColumn) {
|
||||||
this.logger?.warn('KTX scan connector does not support column sampling; using available metadata only', {
|
this.logger?.warn('ktx scan connector does not support column sampling; using available metadata only', {
|
||||||
connectorId: input.connector.id,
|
connectorId: input.connector.id,
|
||||||
table: input.table.name,
|
table: input.table.name,
|
||||||
column: column.name,
|
column: column.name,
|
||||||
|
|
|
||||||
|
|
@ -154,7 +154,7 @@ function scanReportPath(connectionId: string, syncId: string): string {
|
||||||
|
|
||||||
function assertSupportedMode(mode: KtxScanMode): void {
|
function assertSupportedMode(mode: KtxScanMode): void {
|
||||||
if (mode !== 'structural' && mode !== 'relationships' && mode !== 'enriched') {
|
if (mode !== 'structural' && mode !== 'relationships' && mode !== 'enriched') {
|
||||||
throw new Error(`Unsupported KTX scan mode: ${mode}`);
|
throw new Error(`Unsupported ktx scan mode: ${mode}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -544,7 +544,7 @@ export async function runLocalScan(options: RunLocalScanOptions): Promise<LocalS
|
||||||
}
|
}
|
||||||
report.warnings.push({
|
report.warnings.push({
|
||||||
code: 'enrichment_failed',
|
code: 'enrichment_failed',
|
||||||
message: `KTX scan enrichment failed after structural scan completed: ${message}`,
|
message: `ktx scan enrichment failed after structural scan completed: ${message}`,
|
||||||
recoverable: true,
|
recoverable: true,
|
||||||
metadata: { mode, detectRelationships: options.detectRelationships ?? false },
|
metadata: { mode, detectRelationships: options.detectRelationships ?? false },
|
||||||
});
|
});
|
||||||
|
|
|
||||||
|
|
@ -55,7 +55,7 @@ function parseWarning(rawWarning: unknown, path: string): KtxScanWarning {
|
||||||
typeof rawWarning.message !== 'string' ||
|
typeof rawWarning.message !== 'string' ||
|
||||||
typeof rawWarning.recoverable !== 'boolean'
|
typeof rawWarning.recoverable !== 'boolean'
|
||||||
) {
|
) {
|
||||||
throw new Error(`Invalid KTX schema warning artifact: ${path}`);
|
throw new Error(`Invalid ktx schema warning artifact: ${path}`);
|
||||||
}
|
}
|
||||||
return {
|
return {
|
||||||
code: rawWarning.code as KtxScanWarning['code'],
|
code: rawWarning.code as KtxScanWarning['code'],
|
||||||
|
|
@ -73,7 +73,7 @@ async function readWarnings(input: ReadLocalScanStructuralSnapshotInput): Promis
|
||||||
const warningRaw = await input.project.fileStore.readFile(path);
|
const warningRaw = await input.project.fileStore.readFile(path);
|
||||||
const parsed = JSON.parse(warningRaw.content) as unknown;
|
const parsed = JSON.parse(warningRaw.content) as unknown;
|
||||||
if (!isRecord(parsed) || !Array.isArray(parsed.warnings)) {
|
if (!isRecord(parsed) || !Array.isArray(parsed.warnings)) {
|
||||||
throw new Error(`Invalid KTX schema warnings artifact: ${path}`);
|
throw new Error(`Invalid ktx schema warnings artifact: ${path}`);
|
||||||
}
|
}
|
||||||
return parsed.warnings.map((warning) => parseWarning(warning, path));
|
return parsed.warnings.map((warning) => parseWarning(warning, path));
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
|
@ -102,7 +102,7 @@ function parseColumn(rawColumn: unknown, path: string): KtxSchemaColumn {
|
||||||
rawColumn.dimensionType !== 'number' &&
|
rawColumn.dimensionType !== 'number' &&
|
||||||
rawColumn.dimensionType !== 'boolean')
|
rawColumn.dimensionType !== 'boolean')
|
||||||
) {
|
) {
|
||||||
throw new Error(`Invalid KTX schema column artifact: ${path}`);
|
throw new Error(`Invalid ktx schema column artifact: ${path}`);
|
||||||
}
|
}
|
||||||
return {
|
return {
|
||||||
name: rawColumn.name,
|
name: rawColumn.name,
|
||||||
|
|
@ -122,7 +122,7 @@ function parseForeignKey(rawForeignKey: unknown, path: string): KtxSchemaForeign
|
||||||
typeof rawForeignKey.toTable !== 'string' ||
|
typeof rawForeignKey.toTable !== 'string' ||
|
||||||
typeof rawForeignKey.toColumn !== 'string'
|
typeof rawForeignKey.toColumn !== 'string'
|
||||||
) {
|
) {
|
||||||
throw new Error(`Invalid KTX schema foreign key artifact: ${path}`);
|
throw new Error(`Invalid ktx schema foreign key artifact: ${path}`);
|
||||||
}
|
}
|
||||||
return {
|
return {
|
||||||
fromColumn: rawForeignKey.fromColumn,
|
fromColumn: rawForeignKey.fromColumn,
|
||||||
|
|
@ -137,7 +137,7 @@ function parseForeignKey(rawForeignKey: unknown, path: string): KtxSchemaForeign
|
||||||
function parseTable(raw: string, path: string): KtxSchemaTable {
|
function parseTable(raw: string, path: string): KtxSchemaTable {
|
||||||
const parsed = JSON.parse(raw) as unknown;
|
const parsed = JSON.parse(raw) as unknown;
|
||||||
if (!isRecord(parsed) || typeof parsed.name !== 'string' || !Array.isArray(parsed.columns)) {
|
if (!isRecord(parsed) || typeof parsed.name !== 'string' || !Array.isArray(parsed.columns)) {
|
||||||
throw new Error(`Invalid KTX schema table artifact: ${path}`);
|
throw new Error(`Invalid ktx schema table artifact: ${path}`);
|
||||||
}
|
}
|
||||||
return {
|
return {
|
||||||
catalog: optionalStringOrNull(parsed.catalog) ?? null,
|
catalog: optionalStringOrNull(parsed.catalog) ?? null,
|
||||||
|
|
|
||||||
|
|
@ -317,7 +317,7 @@ function compositeSkipBlocks(report: KtxRelationshipBenchmarkReport): string[] {
|
||||||
|
|
||||||
export function formatKtxRelationshipBenchmarkReportMarkdown(report: KtxRelationshipBenchmarkReport): string {
|
export function formatKtxRelationshipBenchmarkReportMarkdown(report: KtxRelationshipBenchmarkReport): string {
|
||||||
const lines = [
|
const lines = [
|
||||||
'# KTX Relationship Discovery Benchmark Evidence',
|
'# ktx Relationship Discovery Benchmark Evidence',
|
||||||
'',
|
'',
|
||||||
`Generated: ${report.generatedAt}`,
|
`Generated: ${report.generatedAt}`,
|
||||||
'',
|
'',
|
||||||
|
|
|
||||||
|
|
@ -153,7 +153,7 @@ async function detectCompositeRelationships(input: {
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
input.warnings.push({
|
input.warnings.push({
|
||||||
code: 'relationship_validation_failed',
|
code: 'relationship_validation_failed',
|
||||||
message: `KTX composite relationship detection failed: ${error instanceof Error ? error.message : String(error)}`,
|
message: `ktx composite relationship detection failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||||
recoverable: true,
|
recoverable: true,
|
||||||
metadata: { source: 'composite_relationship_detection' },
|
metadata: { source: 'composite_relationship_detection' },
|
||||||
});
|
});
|
||||||
|
|
@ -185,7 +185,7 @@ function sqlExecutor(input: DiscoverKtxRelationshipsInput): {
|
||||||
warnings: [
|
warnings: [
|
||||||
{
|
{
|
||||||
code: 'connector_capability_missing',
|
code: 'connector_capability_missing',
|
||||||
message: 'KTX scan connector cannot run read-only SQL relationship validation',
|
message: 'ktx scan connector cannot run read-only SQL relationship validation',
|
||||||
recoverable: true,
|
recoverable: true,
|
||||||
metadata: { capability: 'readOnlySql' },
|
metadata: { capability: 'readOnlySql' },
|
||||||
},
|
},
|
||||||
|
|
@ -199,7 +199,7 @@ function sqlExecutor(input: DiscoverKtxRelationshipsInput): {
|
||||||
warnings: [
|
warnings: [
|
||||||
{
|
{
|
||||||
code: 'relationship_validation_failed',
|
code: 'relationship_validation_failed',
|
||||||
message: 'KTX scan connector advertises readOnlySql but does not expose executeReadOnly',
|
message: 'ktx scan connector advertises readOnlySql but does not expose executeReadOnly',
|
||||||
recoverable: true,
|
recoverable: true,
|
||||||
metadata: { capability: 'readOnlySql' },
|
metadata: { capability: 'readOnlySql' },
|
||||||
},
|
},
|
||||||
|
|
|
||||||
|
|
@ -182,7 +182,7 @@ function mapValidProposals(
|
||||||
const toColumn = toTable ? findColumn(toTable, item.toColumn) : null;
|
const toColumn = toTable ? findColumn(toTable, item.toColumn) : null;
|
||||||
if (!fromTable || !toTable || !fromColumn || !toColumn) {
|
if (!fromTable || !toTable || !fromColumn || !toColumn) {
|
||||||
warnings.push(
|
warnings.push(
|
||||||
invalidReferenceWarning('KTX relationship LLM proposal referenced a table or column that is not in the schema.', {
|
invalidReferenceWarning('ktx relationship LLM proposal referenced a table or column that is not in the schema.', {
|
||||||
proposal: item,
|
proposal: item,
|
||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
|
|
@ -218,7 +218,7 @@ function generationFailureWarning(error: unknown): KtxScanWarning {
|
||||||
const message = error instanceof Error ? error.message : String(error);
|
const message = error instanceof Error ? error.message : String(error);
|
||||||
return {
|
return {
|
||||||
code: 'relationship_llm_proposal_failed',
|
code: 'relationship_llm_proposal_failed',
|
||||||
message: `KTX relationship LLM proposal failed: ${message}`,
|
message: `ktx relationship LLM proposal failed: ${message}`,
|
||||||
recoverable: true,
|
recoverable: true,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
@ -233,7 +233,7 @@ export async function proposeKtxRelationshipCandidatesWithLlm(
|
||||||
const settings = mergeSettings(input.settings);
|
const settings = mergeSettings(input.settings);
|
||||||
const evidence = buildEvidencePacket(input.schema, input.profile, settings);
|
const evidence = buildEvidencePacket(input.schema, input.profile, settings);
|
||||||
const system = [
|
const system = [
|
||||||
'You are helping KTX review possible SQL relationships before validation.',
|
'You are helping ktx review possible SQL relationships before validation.',
|
||||||
'Use only the compact schema evidence. Propose likely primary keys and foreign keys for later SQL validation.',
|
'Use only the compact schema evidence. Propose likely primary keys and foreign keys for later SQL validation.',
|
||||||
'Return structured output only; never assume a join is accepted.',
|
'Return structured output only; never assume a join is accepted.',
|
||||||
].join('\n');
|
].join('\n');
|
||||||
|
|
|
||||||
|
|
@ -115,7 +115,7 @@ export class SemanticLayerService {
|
||||||
async listConnectionIds(): Promise<string[]> {
|
async listConnectionIds(): Promise<string[]> {
|
||||||
try {
|
try {
|
||||||
const result = await this.configService.listFiles(SL_DIR_PREFIX);
|
const result = await this.configService.listFiles(SL_DIR_PREFIX);
|
||||||
// Directories under semantic-layer/ are connectionIds. Local KTX projects use
|
// Directories under semantic-layer/ are connectionIds. Local ktx projects use
|
||||||
// readable ids like "warehouse" and "dbt-main", not only UUIDs.
|
// readable ids like "warehouse" and "dbt-main", not only UUIDs.
|
||||||
return result.files
|
return result.files
|
||||||
.map((f) => f.replace(`${SL_DIR_PREFIX}/`, '').split('/')[0])
|
.map((f) => f.replace(`${SL_DIR_PREFIX}/`, '').split('/')[0])
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
import type { ZodType } from 'zod';
|
import type { z } from 'zod';
|
||||||
import type { GitAuthorResolverPort } from '../../../context/tools/authors.js';
|
import type { GitAuthorResolverPort } from '../../../context/tools/authors.js';
|
||||||
import type { ToolContext, ToolOutput } from '../../../context/tools/base-tool.js';
|
import type { ToolContext, ToolOutput } from '../../../context/tools/base-tool.js';
|
||||||
import { BaseTool } from '../../../context/tools/base-tool.js';
|
import { BaseTool } from '../../../context/tools/base-tool.js';
|
||||||
|
|
@ -27,7 +27,9 @@ export interface BaseSemanticLayerToolDeps {
|
||||||
|
|
||||||
// ── Abstract base class ──
|
// ── Abstract base class ──
|
||||||
|
|
||||||
export abstract class BaseSemanticLayerTool<TInput extends ZodType = ZodType> extends BaseTool<TInput> {
|
export abstract class BaseSemanticLayerTool<
|
||||||
|
TInput extends z.ZodObject<z.ZodRawShape> = z.ZodObject<z.ZodRawShape>,
|
||||||
|
> extends BaseTool<TInput> {
|
||||||
protected readonly semanticLayerService: SemanticLayerService;
|
protected readonly semanticLayerService: SemanticLayerService;
|
||||||
protected readonly slSearchService: SlSearchService;
|
protected readonly slSearchService: SlSearchService;
|
||||||
protected readonly authorResolver: GitAuthorResolverPort;
|
protected readonly authorResolver: GitAuthorResolverPort;
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
import { tool } from 'ai';
|
import { tool, type Tool } from 'ai';
|
||||||
import { z, type ZodType } from 'zod';
|
import { z } from 'zod';
|
||||||
import { noopLogger, type KtxLogger } from '../../context/core/config.js';
|
import { noopLogger, type KtxLogger } from '../../context/core/config.js';
|
||||||
import type { KtxRuntimeToolDescriptor } from '../llm/runtime-port.js';
|
import type { KtxRuntimeToolDescriptor } from '../llm/runtime-port.js';
|
||||||
import { normalizeKtxRuntimeToolOutput } from '../llm/runtime-tools.js';
|
import { normalizeKtxRuntimeToolOutput } from '../llm/runtime-tools.js';
|
||||||
|
|
@ -73,7 +73,7 @@ interface MethodologyEntry {
|
||||||
/**
|
/**
|
||||||
* SECURITY: All tools require authentication. userId must always be provided in ToolContext.
|
* SECURITY: All tools require authentication. userId must always be provided in ToolContext.
|
||||||
*/
|
*/
|
||||||
export abstract class BaseTool<TInput extends ZodType = ZodType> {
|
export abstract class BaseTool<TInput extends z.ZodObject<z.ZodRawShape> = z.ZodObject<z.ZodRawShape>> {
|
||||||
protected readonly logger: KtxLogger;
|
protected readonly logger: KtxLogger;
|
||||||
|
|
||||||
abstract readonly name: string;
|
abstract readonly name: string;
|
||||||
|
|
@ -86,37 +86,9 @@ export abstract class BaseTool<TInput extends ZodType = ZodType> {
|
||||||
|
|
||||||
abstract get inputSchema(): TInput;
|
abstract get inputSchema(): TInput;
|
||||||
|
|
||||||
abstract call(input: z.infer<TInput>, context: ToolContext): Promise<any>;
|
abstract call(input: z.infer<TInput>, context: ToolContext): Promise<unknown>;
|
||||||
|
|
||||||
getParametersSchema(): {
|
toAiSdkTool(context: ToolContext): Tool {
|
||||||
type: 'object';
|
|
||||||
properties: Record<string, any>;
|
|
||||||
required?: string[];
|
|
||||||
} {
|
|
||||||
const jsonSchema = z.toJSONSchema(this.inputSchema, {
|
|
||||||
target: 'draft-7',
|
|
||||||
});
|
|
||||||
|
|
||||||
return jsonSchema as any;
|
|
||||||
}
|
|
||||||
|
|
||||||
toAnthropicFormat(): {
|
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
input_schema: {
|
|
||||||
type: 'object';
|
|
||||||
properties: Record<string, any>;
|
|
||||||
required?: string[];
|
|
||||||
};
|
|
||||||
} {
|
|
||||||
return {
|
|
||||||
name: this.name,
|
|
||||||
description: this.description,
|
|
||||||
input_schema: this.getParametersSchema(),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
toAiSdkTool(context: ToolContext): any {
|
|
||||||
const toolName = this.name;
|
const toolName = this.name;
|
||||||
const logger = this.logger;
|
const logger = this.logger;
|
||||||
|
|
||||||
|
|
@ -137,7 +109,7 @@ export abstract class BaseTool<TInput extends ZodType = ZodType> {
|
||||||
if (!callContext.userId) {
|
if (!callContext.userId) {
|
||||||
throw new Error('Authentication required: userId must be provided in ToolContext');
|
throw new Error('Authentication required: userId must be provided in ToolContext');
|
||||||
}
|
}
|
||||||
const parsedInput = this.parseInput(params as Record<string, any>);
|
const parsedInput = this.parseInput(params as Record<string, unknown>);
|
||||||
const result = await this.call(parsedInput, callContext);
|
const result = await this.call(parsedInput, callContext);
|
||||||
return result;
|
return result;
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
|
@ -171,19 +143,19 @@ export abstract class BaseTool<TInput extends ZodType = ZodType> {
|
||||||
return {
|
return {
|
||||||
name: toolName,
|
name: toolName,
|
||||||
description: this.description,
|
description: this.description,
|
||||||
inputSchema: this.inputSchema as unknown as KtxRuntimeToolDescriptor['inputSchema'],
|
inputSchema: this.inputSchema,
|
||||||
execute: async (params) => {
|
execute: async (params) => {
|
||||||
const callContext = { ...context };
|
const callContext = { ...context };
|
||||||
if (!callContext.userId) {
|
if (!callContext.userId) {
|
||||||
throw new Error('Authentication required: userId must be provided in ToolContext');
|
throw new Error('Authentication required: userId must be provided in ToolContext');
|
||||||
}
|
}
|
||||||
const parsedInput = this.parseInput(params as Record<string, any>);
|
const parsedInput = this.parseInput(params as Record<string, unknown>);
|
||||||
return normalizeKtxRuntimeToolOutput(await this.call(parsedInput, callContext));
|
return normalizeKtxRuntimeToolOutput(await this.call(parsedInput, callContext));
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
parseInput(input: Record<string, any>): z.infer<TInput> {
|
parseInput(input: Record<string, unknown>): z.infer<TInput> {
|
||||||
return this.inputSchema.parse(input);
|
return this.inputSchema.parse(input);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -482,7 +482,7 @@ export function renderInvalidConfigMessage(
|
||||||
const abbreviated = abbreviateHome(projectDir) ?? projectDir;
|
const abbreviated = abbreviateHome(projectDir) ?? projectDir;
|
||||||
|
|
||||||
const lines: string[] = [];
|
const lines: string[] = [];
|
||||||
lines.push(`${bold('KTX status')} ${dim('·')} ${abbreviated}`);
|
lines.push(`${bold('ktx status')} ${dim('·')} ${abbreviated}`);
|
||||||
lines.push('');
|
lines.push('');
|
||||||
lines.push(` ${status('fail', '✗')} ${bold('Config')} ktx.yaml has ${issues.length} schema issue${issues.length === 1 ? '' : 's'}`);
|
lines.push(` ${status('fail', '✗')} ${bold('Config')} ktx.yaml has ${issues.length} schema issue${issues.length === 1 ? '' : 's'}`);
|
||||||
for (const issue of issues) {
|
for (const issue of issues) {
|
||||||
|
|
@ -524,7 +524,7 @@ export function renderValidConfigMessage(
|
||||||
const abbreviated = abbreviateHome(projectDir) ?? projectDir;
|
const abbreviated = abbreviateHome(projectDir) ?? projectDir;
|
||||||
|
|
||||||
const lines: string[] = [];
|
const lines: string[] = [];
|
||||||
lines.push(`${bold('KTX status')} ${dim('·')} ${abbreviated}`);
|
lines.push(`${bold('ktx status')} ${dim('·')} ${abbreviated}`);
|
||||||
lines.push('');
|
lines.push('');
|
||||||
lines.push(` ${status('pass', '✓')} ${bold('Config')} ${dim('ktx.yaml schema valid')}`);
|
lines.push(` ${status('pass', '✓')} ${bold('Config')} ${dim('ktx.yaml schema valid')}`);
|
||||||
lines.push('');
|
lines.push('');
|
||||||
|
|
@ -559,9 +559,9 @@ export function renderMissingProjectMessage(
|
||||||
const envProjectDir = process.env.KTX_PROJECT_DIR;
|
const envProjectDir = process.env.KTX_PROJECT_DIR;
|
||||||
|
|
||||||
const lines: string[] = [];
|
const lines: string[] = [];
|
||||||
lines.push(`${bold('KTX status')} ${dim('·')} ${abbreviated}`);
|
lines.push(`${bold('ktx status')} ${dim('·')} ${abbreviated}`);
|
||||||
lines.push('');
|
lines.push('');
|
||||||
lines.push(` No KTX project here yet. ${dim('(ktx.yaml not found)')}`);
|
lines.push(` No ktx project here yet. ${dim('(ktx.yaml not found)')}`);
|
||||||
lines.push('');
|
lines.push('');
|
||||||
lines.push(` Run ${bold('ktx setup')} to create one.`);
|
lines.push(` Run ${bold('ktx setup')} to create one.`);
|
||||||
if (envProjectDir !== undefined) {
|
if (envProjectDir !== undefined) {
|
||||||
|
|
@ -643,7 +643,7 @@ export async function runKtxDoctor(
|
||||||
}
|
}
|
||||||
|
|
||||||
const setupChecks = await runSetupChecks();
|
const setupChecks = await runSetupChecks();
|
||||||
const report: DoctorReport = { title: 'KTX status', checks: setupChecks };
|
const report: DoctorReport = { title: 'ktx status', checks: setupChecks };
|
||||||
const renderOptions: RenderOptions = {
|
const renderOptions: RenderOptions = {
|
||||||
verbose: args.verbose ?? false,
|
verbose: args.verbose ?? false,
|
||||||
useColor: shouldUseColorOutput(io.stdout),
|
useColor: shouldUseColorOutput(io.stdout),
|
||||||
|
|
|
||||||
|
|
@ -38,7 +38,7 @@ export async function runKtxEmbeddingHealthCheck(
|
||||||
try {
|
try {
|
||||||
const provider = createKtxEmbeddingProvider(config, options.deps);
|
const provider = createKtxEmbeddingProvider(config, options.deps);
|
||||||
const embedding = await withTimeout(
|
const embedding = await withTimeout(
|
||||||
provider.embed(options.text ?? 'KTX embedding health check'),
|
provider.embed(options.text ?? 'ktx embedding health check'),
|
||||||
options.timeoutMs ?? 15_000,
|
options.timeoutMs ?? 15_000,
|
||||||
);
|
);
|
||||||
if (embedding.length !== config.dimensions) {
|
if (embedding.length !== config.dimensions) {
|
||||||
|
|
|
||||||
|
|
@ -77,7 +77,7 @@ class OpenAIEmbeddingProvider implements KtxEmbeddingProvider {
|
||||||
this.dimensions = config.dimensions;
|
this.dimensions = config.dimensions;
|
||||||
this.maxBatchSize = config.batchSize ?? DEFAULT_BATCH_SIZE;
|
this.maxBatchSize = config.batchSize ?? DEFAULT_BATCH_SIZE;
|
||||||
if (!config.openai?.apiKey) {
|
if (!config.openai?.apiKey) {
|
||||||
throw new Error('openai.apiKey is required when KTX embedding backend is openai');
|
throw new Error('openai.apiKey is required when ktx embedding backend is openai');
|
||||||
}
|
}
|
||||||
this.client = deps.createOpenAIClient
|
this.client = deps.createOpenAIClient
|
||||||
? deps.createOpenAIClient({ apiKey: config.openai.apiKey, baseURL: config.openai.baseURL })
|
? deps.createOpenAIClient({ apiKey: config.openai.apiKey, baseURL: config.openai.baseURL })
|
||||||
|
|
@ -122,7 +122,7 @@ class SentenceTransformersEmbeddingProvider implements KtxEmbeddingProvider {
|
||||||
|
|
||||||
constructor(config: KtxEmbeddingConfig, deps: KtxEmbeddingProviderDeps) {
|
constructor(config: KtxEmbeddingConfig, deps: KtxEmbeddingProviderDeps) {
|
||||||
if (!config.sentenceTransformers?.baseURL) {
|
if (!config.sentenceTransformers?.baseURL) {
|
||||||
throw new Error('sentenceTransformers.baseURL is required when KTX embedding backend is sentence-transformers');
|
throw new Error('sentenceTransformers.baseURL is required when ktx embedding backend is sentence-transformers');
|
||||||
}
|
}
|
||||||
this.dimensions = config.dimensions;
|
this.dimensions = config.dimensions;
|
||||||
this.maxBatchSize = config.batchSize ?? DEFAULT_BATCH_SIZE;
|
this.maxBatchSize = config.batchSize ?? DEFAULT_BATCH_SIZE;
|
||||||
|
|
@ -207,6 +207,6 @@ export function createKtxEmbeddingProvider(
|
||||||
case 'sentence-transformers':
|
case 'sentence-transformers':
|
||||||
return new SentenceTransformersEmbeddingProvider(config, deps);
|
return new SentenceTransformersEmbeddingProvider(config, deps);
|
||||||
default:
|
default:
|
||||||
throw new Error(`Unsupported KTX embedding backend: ${String((config as { backend?: string }).backend)}`);
|
throw new Error(`Unsupported ktx embedding backend: ${String((config as { backend?: string }).backend)}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -166,7 +166,7 @@ class DefaultKtxLlmProvider implements KtxLlmProvider {
|
||||||
|
|
||||||
if (config.backend === 'vertex') {
|
if (config.backend === 'vertex') {
|
||||||
if (!config.vertex?.location) {
|
if (!config.vertex?.location) {
|
||||||
throw new Error('vertex.location is required when KTX LLM backend is vertex');
|
throw new Error('vertex.location is required when ktx LLM backend is vertex');
|
||||||
}
|
}
|
||||||
const vertex = (deps.createVertexAnthropic ?? createVertexAnthropic)({
|
const vertex = (deps.createVertexAnthropic ?? createVertexAnthropic)({
|
||||||
...(config.vertex.project ? { project: config.vertex.project } : {}),
|
...(config.vertex.project ? { project: config.vertex.project } : {}),
|
||||||
|
|
|
||||||
|
|
@ -25,7 +25,7 @@ export async function createKtxCliScanConnector(
|
||||||
const registration = getDriverRegistration(driver);
|
const registration = getDriverRegistration(driver);
|
||||||
if (!registration) {
|
if (!registration) {
|
||||||
throw new Error(
|
throw new Error(
|
||||||
`Connection "${connectionId}" uses driver "${driver}", which has no native standalone KTX scan connector. Supported drivers: ${SUPPORTED_DRIVERS}.`,
|
`Connection "${connectionId}" uses driver "${driver}", which has no native standalone ktx scan connector. Supported drivers: ${SUPPORTED_DRIVERS}.`,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -74,7 +74,7 @@ export async function ensureManagedLocalEmbeddingsDaemon(
|
||||||
});
|
});
|
||||||
|
|
||||||
const verb = daemon.status === 'started' ? 'Started' : 'Using';
|
const verb = daemon.status === 'started' ? 'Started' : 'Using';
|
||||||
writePrefixedLines((chunk) => options.io.stderr.write(chunk), `${verb} KTX daemon: ${daemon.baseUrl}`);
|
writePrefixedLines((chunk) => options.io.stderr.write(chunk), `${verb} ktx daemon: ${daemon.baseUrl}`);
|
||||||
|
|
||||||
return {
|
return {
|
||||||
baseUrl: daemon.baseUrl,
|
baseUrl: daemon.baseUrl,
|
||||||
|
|
|
||||||
|
|
@ -139,7 +139,7 @@ export async function startKtxMcpDaemon(options: {
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
throw new Error(
|
throw new Error(
|
||||||
`KTX MCP daemon is already running at http://${existing.host}:${existing.port}/mcp ` +
|
`ktx MCP daemon is already running at http://${existing.host}:${existing.port}/mcp ` +
|
||||||
'with a different configuration. Run `ktx mcp stop` first, then start again.',
|
'with a different configuration. Run `ktx mcp stop` first, then start again.',
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
@ -175,7 +175,7 @@ export async function startKtxMcpDaemon(options: {
|
||||||
}),
|
}),
|
||||||
});
|
});
|
||||||
if (!child.pid) {
|
if (!child.pid) {
|
||||||
throw new Error('Failed to start KTX MCP daemon: child process pid was not available.');
|
throw new Error('Failed to start ktx MCP daemon: child process pid was not available.');
|
||||||
}
|
}
|
||||||
child.unref();
|
child.unref();
|
||||||
const state: KtxMcpDaemonState = {
|
const state: KtxMcpDaemonState = {
|
||||||
|
|
@ -219,7 +219,7 @@ export async function readKtxMcpDaemonStatus(options: {
|
||||||
}
|
}
|
||||||
return {
|
return {
|
||||||
kind: 'running',
|
kind: 'running',
|
||||||
detail: `KTX MCP daemon running at http://${state.host}:${state.port}/mcp`,
|
detail: `ktx MCP daemon running at http://${state.host}:${state.port}/mcp`,
|
||||||
state,
|
state,
|
||||||
url: `http://${state.host}:${state.port}/mcp`,
|
url: `http://${state.host}:${state.port}/mcp`,
|
||||||
};
|
};
|
||||||
|
|
|
||||||
|
|
@ -62,11 +62,11 @@ export function managedRuntimeInstallCommand(feature: KtxRuntimeFeature): string
|
||||||
|
|
||||||
function installPrompt(feature: KtxRuntimeFeature): string {
|
function installPrompt(feature: KtxRuntimeFeature): string {
|
||||||
const label = feature === 'local-embeddings' ? 'local embeddings Python runtime' : 'core Python runtime';
|
const label = feature === 'local-embeddings' ? 'local embeddings Python runtime' : 'core Python runtime';
|
||||||
return `KTX needs to install the ${label}. This downloads Python dependencies with uv. Continue?`;
|
return `ktx needs to install the ${label}. This downloads Python dependencies with uv. Continue?`;
|
||||||
}
|
}
|
||||||
|
|
||||||
function runtimeRequiredMessage(feature: KtxRuntimeFeature): string {
|
function runtimeRequiredMessage(feature: KtxRuntimeFeature): string {
|
||||||
return `KTX Python runtime is required for this command. Run: ${managedRuntimeInstallCommand(feature)}`;
|
return `ktx Python runtime is required for this command. Run: ${managedRuntimeInstallCommand(feature)}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
function hasFeature(manifest: InstalledKtxRuntimeManifest, feature: KtxRuntimeFeature): boolean {
|
function hasFeature(manifest: InstalledKtxRuntimeManifest, feature: KtxRuntimeFeature): boolean {
|
||||||
|
|
@ -101,22 +101,22 @@ export async function ensureManagedPythonCommandRuntime(
|
||||||
const confirmInstall = options.confirmInstall ?? defaultConfirmInstall;
|
const confirmInstall = options.confirmInstall ?? defaultConfirmInstall;
|
||||||
const confirmed = await confirmInstall(installPrompt(feature), options.io);
|
const confirmed = await confirmInstall(installPrompt(feature), options.io);
|
||||||
if (!confirmed) {
|
if (!confirmed) {
|
||||||
throw new Error(`KTX Python runtime installation was cancelled. Run: ${managedRuntimeInstallCommand(feature)}`);
|
throw new Error(`ktx Python runtime installation was cancelled. Run: ${managedRuntimeInstallCommand(feature)}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const progress = (options.spinner ?? (() => createStaticCliSpinner(options.io)))();
|
const progress = (options.spinner ?? (() => createStaticCliSpinner(options.io)))();
|
||||||
progress.start(`Installing KTX Python runtime (${feature}) with uv...`);
|
progress.start(`Installing ktx Python runtime (${feature}) with uv...`);
|
||||||
try {
|
try {
|
||||||
const installed = await installRuntime({
|
const installed = await installRuntime({
|
||||||
cliVersion: options.cliVersion,
|
cliVersion: options.cliVersion,
|
||||||
features: [feature],
|
features: [feature],
|
||||||
force: false,
|
force: false,
|
||||||
});
|
});
|
||||||
progress.stop(`KTX Python runtime ready: ${installed.layout.versionDir}`);
|
progress.stop(`ktx Python runtime ready: ${installed.layout.versionDir}`);
|
||||||
return { layout: installed.layout, manifest: installed.manifest };
|
return { layout: installed.layout, manifest: installed.manifest };
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
progress.error(`KTX Python runtime install failed: ${error instanceof Error ? error.message : String(error)}`);
|
progress.error(`ktx Python runtime install failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -21,7 +21,7 @@ export class ManagedPythonDaemonStartError extends Error {
|
||||||
readonly detail: string;
|
readonly detail: string;
|
||||||
readonly stderrLog: string;
|
readonly stderrLog: string;
|
||||||
constructor(detail: string, stderrLog: string) {
|
constructor(detail: string, stderrLog: string) {
|
||||||
super(`KTX daemon failed to start: ${detail}. stderr: ${stderrLog}`);
|
super(`ktx daemon failed to start: ${detail}. stderr: ${stderrLog}`);
|
||||||
this.name = 'ManagedPythonDaemonStartError';
|
this.name = 'ManagedPythonDaemonStartError';
|
||||||
this.detail = detail;
|
this.detail = detail;
|
||||||
this.stderrLog = stderrLog;
|
this.stderrLog = stderrLog;
|
||||||
|
|
@ -720,7 +720,7 @@ export async function startManagedPythonDaemon(
|
||||||
);
|
);
|
||||||
child.unref();
|
child.unref();
|
||||||
if (!child.pid) {
|
if (!child.pid) {
|
||||||
throw new Error(`KTX daemon did not report a pid. stderr: ${layout.daemonStderrPath}`);
|
throw new Error(`ktx daemon did not report a pid. stderr: ${layout.daemonStderrPath}`);
|
||||||
}
|
}
|
||||||
const state: ManagedPythonDaemonState = {
|
const state: ManagedPythonDaemonState = {
|
||||||
schemaVersion: 1,
|
schemaVersion: 1,
|
||||||
|
|
|
||||||
|
|
@ -65,7 +65,7 @@ function normalizedBaseUrl(baseUrl: string): string {
|
||||||
function parseJsonObject(raw: string, path: string): Record<string, unknown> {
|
function parseJsonObject(raw: string, path: string): Record<string, unknown> {
|
||||||
const parsed = JSON.parse(raw) as unknown;
|
const parsed = JSON.parse(raw) as unknown;
|
||||||
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
|
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
|
||||||
throw new Error(`KTX daemon HTTP ${path} returned non-object JSON`);
|
throw new Error(`ktx daemon HTTP ${path} returned non-object JSON`);
|
||||||
}
|
}
|
||||||
return parsed as Record<string, unknown>;
|
return parsed as Record<string, unknown>;
|
||||||
}
|
}
|
||||||
|
|
@ -96,7 +96,7 @@ async function postManagedDaemonJson(
|
||||||
const text = Buffer.concat(chunks).toString('utf8');
|
const text = Buffer.concat(chunks).toString('utf8');
|
||||||
const statusCode = response.statusCode ?? 0;
|
const statusCode = response.statusCode ?? 0;
|
||||||
if (statusCode < 200 || statusCode >= 300) {
|
if (statusCode < 200 || statusCode >= 300) {
|
||||||
reject(new Error(`KTX daemon HTTP ${path} failed with ${statusCode}: ${text}`));
|
reject(new Error(`ktx daemon HTTP ${path} failed with ${statusCode}: ${text}`));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
|
|
@ -138,7 +138,7 @@ export function createManagedPythonDaemonBaseUrlResolver(
|
||||||
force: false,
|
force: false,
|
||||||
});
|
});
|
||||||
const verb = daemon.status === 'started' ? 'Started' : 'Using existing';
|
const verb = daemon.status === 'started' ? 'Started' : 'Using existing';
|
||||||
writePrefixedLines((chunk) => options.io.stderr.write(chunk), `${verb} KTX daemon: ${daemon.baseUrl}`);
|
writePrefixedLines((chunk) => options.io.stderr.write(chunk), `${verb} ktx daemon: ${daemon.baseUrl}`);
|
||||||
cachedBaseUrl = daemon.baseUrl;
|
cachedBaseUrl = daemon.baseUrl;
|
||||||
return cachedBaseUrl;
|
return cachedBaseUrl;
|
||||||
};
|
};
|
||||||
|
|
|
||||||
|
|
@ -124,7 +124,7 @@ export interface ManagedPythonRuntimeDoctorCheck {
|
||||||
|
|
||||||
/** @internal */
|
/** @internal */
|
||||||
export const MISSING_UV_RUNTIME_INSTALL_MESSAGE =
|
export const MISSING_UV_RUNTIME_INSTALL_MESSAGE =
|
||||||
'uv is required to install the KTX Python runtime. KTX does not download uv automatically. Install uv, make sure it is on PATH, and retry: ktx admin runtime install --yes';
|
'uv is required to install the ktx Python runtime. ktx does not download uv automatically. Install uv, make sure it is on PATH, and retry: ktx admin runtime install --yes';
|
||||||
|
|
||||||
function defaultAssetDir(): string {
|
function defaultAssetDir(): string {
|
||||||
return fileURLToPath(new URL('../assets/python/', import.meta.url));
|
return fileURLToPath(new URL('../assets/python/', import.meta.url));
|
||||||
|
|
@ -250,7 +250,7 @@ export async function verifyRuntimeAsset(input: { assetDir: string }): Promise<M
|
||||||
[
|
[
|
||||||
`Missing bundled Python runtime manifest: ${manifestPath}`,
|
`Missing bundled Python runtime manifest: ${manifestPath}`,
|
||||||
'In a source checkout, build the local runtime assets with: pnpm run artifacts:build',
|
'In a source checkout, build the local runtime assets with: pnpm run artifacts:build',
|
||||||
'Then retry the runtime-backed KTX command.',
|
'Then retry the runtime-backed ktx command.',
|
||||||
].join('\n'),
|
].join('\n'),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -64,7 +64,7 @@ function fullOrigin(value: string): string {
|
||||||
|
|
||||||
export function buildMcpSecurityConfig(input: McpSecurityConfigInput): McpSecurityConfig {
|
export function buildMcpSecurityConfig(input: McpSecurityConfigInput): McpSecurityConfig {
|
||||||
if (!isLoopbackHost(input.host) && !input.token) {
|
if (!isLoopbackHost(input.host) && !input.token) {
|
||||||
throw new Error(`Binding KTX MCP to ${input.host} requires --token or KTX_MCP_TOKEN`);
|
throw new Error(`Binding ktx MCP to ${input.host} requires --token or KTX_MCP_TOKEN`);
|
||||||
}
|
}
|
||||||
const allowedHostSet = new Set<string>(DEFAULT_ALLOWED_HOSTS);
|
const allowedHostSet = new Set<string>(DEFAULT_ALLOWED_HOSTS);
|
||||||
if (!isLoopbackHost(input.host)) {
|
if (!isLoopbackHost(input.host)) {
|
||||||
|
|
@ -94,16 +94,16 @@ export function isMcpRequestAuthorized(
|
||||||
): McpAuthorizationResult {
|
): McpAuthorizationResult {
|
||||||
const host = headerValue(request.headers, 'host');
|
const host = headerValue(request.headers, 'host');
|
||||||
if (!host || !config.allowedHosts.includes(normalizeHostHeader(host))) {
|
if (!host || !config.allowedHosts.includes(normalizeHostHeader(host))) {
|
||||||
return { ok: false, status: 403, message: 'Host header is not allowed for KTX MCP.' };
|
return { ok: false, status: 403, message: 'Host header is not allowed for ktx MCP.' };
|
||||||
}
|
}
|
||||||
const origin = headerValue(request.headers, 'origin');
|
const origin = headerValue(request.headers, 'origin');
|
||||||
if (origin && !config.allowedOrigins.includes(origin)) {
|
if (origin && !config.allowedOrigins.includes(origin)) {
|
||||||
return { ok: false, status: 403, message: 'Origin header is not allowed for KTX MCP.' };
|
return { ok: false, status: 403, message: 'Origin header is not allowed for ktx MCP.' };
|
||||||
}
|
}
|
||||||
if (request.path === '/mcp' && config.token) {
|
if (request.path === '/mcp' && config.token) {
|
||||||
const auth = headerValue(request.headers, 'authorization');
|
const auth = headerValue(request.headers, 'authorization');
|
||||||
if (auth !== `Bearer ${config.token}`) {
|
if (auth !== `Bearer ${config.token}`) {
|
||||||
return { ok: false, status: 401, message: 'Missing or invalid KTX MCP bearer token.' };
|
return { ok: false, status: 401, message: 'Missing or invalid ktx MCP bearer token.' };
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return { ok: true };
|
return { ok: true };
|
||||||
|
|
|
||||||
|
|
@ -65,7 +65,7 @@ export async function createKtxMcpServerFactory(input: {
|
||||||
embeddingProvider,
|
embeddingProvider,
|
||||||
});
|
});
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
io.stderr.write(`KTX MCP memory_ingest disabled: ${error instanceof Error ? error.message : String(error)}\n`);
|
io.stderr.write(`ktx MCP memory_ingest disabled: ${error instanceof Error ? error.message : String(error)}\n`);
|
||||||
}
|
}
|
||||||
|
|
||||||
return () =>
|
return () =>
|
||||||
|
|
|
||||||
|
|
@ -52,7 +52,7 @@ export async function runKtxMcpStdioServer(options: RunKtxMcpStdioServerOptions)
|
||||||
};
|
};
|
||||||
transport.onclose = () => settle(resolve);
|
transport.onclose = () => settle(resolve);
|
||||||
transport.onerror = (error) => {
|
transport.onerror = (error) => {
|
||||||
options.io?.stderr.write(`KTX MCP stdio transport error: ${error.message}\n`);
|
options.io?.stderr.write(`ktx MCP stdio transport error: ${error.message}\n`);
|
||||||
settle(() => reject(error));
|
settle(() => reject(error));
|
||||||
};
|
};
|
||||||
stdin.once('end', closeTransport);
|
stdin.once('end', closeTransport);
|
||||||
|
|
|
||||||
|
|
@ -363,7 +363,7 @@ export function ActivityFeed(props: {
|
||||||
</Box>
|
</Box>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
{/* Results — what KTX has created */}
|
{/* Results — what ktx has created */}
|
||||||
{insights.length > 0 && (
|
{insights.length > 0 && (
|
||||||
<Box flexDirection="column" marginTop={1}>
|
<Box flexDirection="column" marginTop={1}>
|
||||||
<Text color={props.theme.text}> Created so far:</Text>
|
<Text color={props.theme.text}> Created so far:</Text>
|
||||||
|
|
@ -395,7 +395,7 @@ export function ActivityFeed(props: {
|
||||||
<Text color={props.theme.active}>{spinner(props.frame)} Saving to context layer...</Text>
|
<Text color={props.theme.active}>{spinner(props.frame)} Saving to context layer...</Text>
|
||||||
)}
|
)}
|
||||||
{savedEvent && (
|
{savedEvent && (
|
||||||
<Text color={props.theme.complete}>✓ Saved — your agents can now use the KTX context layer</Text>
|
<Text color={props.theme.complete}>✓ Saved — your agents can now use the ktx context layer</Text>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
{/* Phase 7: Completion */}
|
{/* Phase 7: Completion */}
|
||||||
|
|
@ -430,12 +430,12 @@ function CompletionSummary(props: {
|
||||||
<>
|
<>
|
||||||
<Text color={props.theme.border}>{'─'.repeat(60)}</Text>
|
<Text color={props.theme.border}>{'─'.repeat(60)}</Text>
|
||||||
<Text bold color={props.theme.complete}>
|
<Text bold color={props.theme.complete}>
|
||||||
★ KTX finished ingesting your data
|
★ ktx finished ingesting your data
|
||||||
</Text>
|
</Text>
|
||||||
{(sl > 0 || wiki > 0) && (
|
{(sl > 0 || wiki > 0) && (
|
||||||
<>
|
<>
|
||||||
<Text />
|
<Text />
|
||||||
<Text color={props.theme.text}>KTX created:</Text>
|
<Text color={props.theme.text}>ktx created:</Text>
|
||||||
{sl > 0 && (
|
{sl > 0 && (
|
||||||
<Text color={props.theme.active}>
|
<Text color={props.theme.active}>
|
||||||
{' '}📊 {sl} query definition{sl === 1 ? '' : 's'} — so agents can write accurate SQL for your data
|
{' '}📊 {sl} query definition{sl === 1 ? '' : 's'} — so agents can write accurate SQL for your data
|
||||||
|
|
|
||||||
|
|
@ -45,7 +45,7 @@ function commandLines(commands: ReadonlyArray<{ command: string; description: st
|
||||||
|
|
||||||
export function formatNextStepLines(indent = ' '): string[] {
|
export function formatNextStepLines(indent = ' '): string[] {
|
||||||
return [
|
return [
|
||||||
`${indent}KTX context is ready for agents. Open your coding agent from the KTX project directory and ask a data question.`,
|
`${indent}ktx context is ready for agents. Open your coding agent from the ktx project directory and ask a data question.`,
|
||||||
`${indent}Verify with:`,
|
`${indent}Verify with:`,
|
||||||
...commandLines(KTX_NEXT_STEP_DIRECT_COMMANDS, indent),
|
...commandLines(KTX_NEXT_STEP_DIRECT_COMMANDS, indent),
|
||||||
];
|
];
|
||||||
|
|
@ -77,7 +77,7 @@ export function formatSetupNextStepLines(state: KtxSetupNextStepState, indent =
|
||||||
|
|
||||||
if (!state.agentIntegrationReady) {
|
if (!state.agentIntegrationReady) {
|
||||||
return [
|
return [
|
||||||
`${indent}KTX context is built. Install agent rules when you want your coding agent to use it.`,
|
`${indent}ktx context is built. Install agent rules when you want your coding agent to use it.`,
|
||||||
`${indent}$ ${'ktx setup --agents'.padEnd(KTX_NEXT_STEP_COMMAND_WIDTH)} Install CLI-based agent rules`,
|
`${indent}$ ${'ktx setup --agents'.padEnd(KTX_NEXT_STEP_COMMAND_WIDTH)} Install CLI-based agent rules`,
|
||||||
`${indent}$ ${'ktx status'.padEnd(KTX_NEXT_STEP_COMMAND_WIDTH)} Check setup and context readiness`,
|
`${indent}$ ${'ktx status'.padEnd(KTX_NEXT_STEP_COMMAND_WIDTH)} Check setup and context readiness`,
|
||||||
];
|
];
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
You are processing ONE WorkUnit of a multi-file ingest bundle. The WorkUnit
|
You are processing ONE WorkUnit of a multi-file ingest bundle. The WorkUnit
|
||||||
gives you a slice of raw source files (LookML views, dbt/MetricFlow YAMLs,
|
gives you a slice of raw source files (LookML views, dbt/MetricFlow YAMLs,
|
||||||
Metabase card JSONs, Notion pages, or similar) and you must translate that
|
Metabase card JSONs, Notion pages, or similar) and you must translate that
|
||||||
slice into KTX semantic-layer sources and/or knowledge wiki pages, in one pass.
|
slice into ktx semantic-layer sources and/or knowledge wiki pages, in one pass.
|
||||||
You run in an isolated WorkUnit worktree. Deterministic projection output,
|
You run in an isolated WorkUnit worktree. Deterministic projection output,
|
||||||
existing project memory, and listed dependency paths are visible; sibling
|
existing project memory, and listed dependency paths are visible; sibling
|
||||||
WorkUnit edits from this same job are not visible until the runner integrates
|
WorkUnit edits from this same job are not visible until the runner integrates
|
||||||
|
|
@ -10,7 +10,7 @@ accepted patches.
|
||||||
</role>
|
</role>
|
||||||
|
|
||||||
<stance>
|
<stance>
|
||||||
Assertive. The bundle was explicitly submitted for ingest. Default to capturing everything the raw files declare that maps cleanly to KTX: one SL source per table/view, one wiki page per non-obvious business rule or alias. Do not abandon a WorkUnit because "some content overlaps with another WU"; use `ingest_triage` to reconcile, do not skip.
|
Assertive. The bundle was explicitly submitted for ingest. Default to capturing everything the raw files declare that maps cleanly to ktx: one SL source per table/view, one wiki page per non-obvious business rule or alias. Do not abandon a WorkUnit because "some content overlaps with another WU"; use `ingest_triage` to reconcile, do not skip.
|
||||||
</stance>
|
</stance>
|
||||||
|
|
||||||
<workflow>
|
<workflow>
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
<role>
|
<role>
|
||||||
You are ingesting an external technical artifact (a LookML view, dbt model, schema description, business glossary, or other reference document) into KTX organizational memory. The user has explicitly submitted this content for bulk ingest. Assume it is intentional and worth capturing.
|
You are ingesting an external technical artifact (a LookML view, dbt model, schema description, business glossary, or other reference document) into ktx organizational memory. The user has explicitly submitted this content for bulk ingest. Assume it is intentional and worth capturing.
|
||||||
</role>
|
</role>
|
||||||
|
|
||||||
<stance>
|
<stance>
|
||||||
|
|
@ -18,7 +18,7 @@ A single artifact typically produces multiple actions: one SL source per table/v
|
||||||
</workflow>
|
</workflow>
|
||||||
|
|
||||||
<scope>
|
<scope>
|
||||||
All wiki writes go to the GLOBAL scope - they will be visible to every user of this KTX project. Phrase wiki pages as objective business knowledge, not personal preference. The `wiki_write` tool handles scope selection automatically for external ingest.
|
All wiki writes go to the GLOBAL scope - they will be visible to every user of this ktx project. Phrase wiki pages as objective business knowledge, not personal preference. The `wiki_write` tool handles scope selection automatically for external ingest.
|
||||||
</scope>
|
</scope>
|
||||||
|
|
||||||
<do_not>
|
<do_not>
|
||||||
|
|
|
||||||
|
|
@ -8,7 +8,7 @@ const DATABASE_INGEST_REPLACEMENTS: Array<[RegExp, string]> = [
|
||||||
[/\bWriting schema artifacts\b/gi, 'Writing schema context'],
|
[/\bWriting schema artifacts\b/gi, 'Writing schema context'],
|
||||||
[/\bEnriching schema metadata\b/gi, 'Building enriched schema context'],
|
[/\bEnriching schema metadata\b/gi, 'Building enriched schema context'],
|
||||||
[
|
[
|
||||||
/\bKTX scan enrichment failed after structural scan completed\b/gi,
|
/\bktx scan enrichment failed after structural scan completed\b/gi,
|
||||||
'Database enrichment failed after schema context completed',
|
'Database enrichment failed after schema context completed',
|
||||||
],
|
],
|
||||||
[/\bstructural scan\b/gi, 'schema context'],
|
[/\bstructural scan\b/gi, 'schema context'],
|
||||||
|
|
|
||||||
|
|
@ -875,8 +875,8 @@ function createPlainPublicIngestProgress(io: KtxCliIo, options: PlainPublicInges
|
||||||
const INTERNAL_STATUS_LINE_RE =
|
const INTERNAL_STATUS_LINE_RE =
|
||||||
/^(Report|Run|Job|Status|Adapter|Connection|Sync|Diff|Tasks|Work units|Failed tasks|Saved memory|Provenance rows):\s*/;
|
/^(Report|Run|Job|Status|Adapter|Connection|Sync|Diff|Tasks|Work units|Failed tasks|Saved memory|Provenance rows):\s*/;
|
||||||
const ACTIONABLE_FAILURE_LINE_RE =
|
const ACTIONABLE_FAILURE_LINE_RE =
|
||||||
/^(Missing bundled Python runtime manifest|KTX Python runtime is required|KTX daemon HTTP|Error:|Failed\b|Could not\b|Cannot\b)/;
|
/^(Missing bundled Python runtime manifest|ktx Python runtime is required|ktx daemon HTTP|Error:|Failed\b|Could not\b|Cannot\b)/;
|
||||||
const RUNTIME_BACKED_RETRY_LINE_RE = /^Then retry the runtime-backed KTX command\.?$/;
|
const RUNTIME_BACKED_RETRY_LINE_RE = /^Then retry the runtime-backed ktx command\.?$/;
|
||||||
|
|
||||||
function trimErrorPrefix(line: string): string {
|
function trimErrorPrefix(line: string): string {
|
||||||
return line.replace(/^Error:\s*/, '');
|
return line.replace(/^Error:\s*/, '');
|
||||||
|
|
@ -887,7 +887,7 @@ function capturedFailureMessage(output: string): string | undefined {
|
||||||
.split(/\r?\n/)
|
.split(/\r?\n/)
|
||||||
.map((line) => line.trim())
|
.map((line) => line.trim())
|
||||||
.filter((line) => line.length > 0)
|
.filter((line) => line.length > 0)
|
||||||
.filter((line) => !line.startsWith('KTX scan completed'))
|
.filter((line) => !line.startsWith('ktx scan completed'))
|
||||||
.filter((line) => !INTERNAL_STATUS_LINE_RE.test(line))
|
.filter((line) => !INTERNAL_STATUS_LINE_RE.test(line))
|
||||||
.map(publicIngestOutputLine);
|
.map(publicIngestOutputLine);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,7 @@ const semverPattern =
|
||||||
|
|
||||||
export function assertCliVersion(value: unknown, source: string): string {
|
export function assertCliVersion(value: unknown, source: string): string {
|
||||||
if (typeof value !== 'string' || !semverPattern.test(value)) {
|
if (typeof value !== 'string' || !semverPattern.test(value)) {
|
||||||
throw new Error(`Invalid KTX CLI version in ${source}`);
|
throw new Error(`Invalid ktx CLI version in ${source}`);
|
||||||
}
|
}
|
||||||
return value;
|
return value;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -90,7 +90,7 @@ export function resolveProjectRuntimeRequirements(
|
||||||
requirements.push({
|
requirements.push({
|
||||||
feature: 'core',
|
feature: 'core',
|
||||||
reason: 'database-introspection',
|
reason: 'database-introspection',
|
||||||
detail: 'Database introspection fallback uses the KTX daemon.',
|
detail: 'Database introspection fallback uses the ktx daemon.',
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -45,7 +45,7 @@ function writeJson(io: KtxCliIo, value: unknown): void {
|
||||||
|
|
||||||
function writeInstallResult(io: KtxCliIo, result: ManagedPythonRuntimeInstallResult): void {
|
function writeInstallResult(io: KtxCliIo, result: ManagedPythonRuntimeInstallResult): void {
|
||||||
const verb = result.status === 'ready' ? 'Using existing' : 'Installed';
|
const verb = result.status === 'ready' ? 'Using existing' : 'Installed';
|
||||||
io.stdout.write(`${verb} KTX Python runtime\n`);
|
io.stdout.write(`${verb} ktx Python runtime\n`);
|
||||||
io.stdout.write(`version: ${result.manifest.cliVersion}\n`);
|
io.stdout.write(`version: ${result.manifest.cliVersion}\n`);
|
||||||
io.stdout.write(`features: ${result.manifest.features.join(', ')}\n`);
|
io.stdout.write(`features: ${result.manifest.features.join(', ')}\n`);
|
||||||
io.stdout.write(`python: ${result.manifest.python.executable}\n`);
|
io.stdout.write(`python: ${result.manifest.python.executable}\n`);
|
||||||
|
|
@ -56,7 +56,7 @@ function writeInstallResult(io: KtxCliIo, result: ManagedPythonRuntimeInstallRes
|
||||||
|
|
||||||
function writeDaemonStart(io: KtxCliIo, result: ManagedPythonDaemonStartResult): void {
|
function writeDaemonStart(io: KtxCliIo, result: ManagedPythonDaemonStartResult): void {
|
||||||
const verb = result.status === 'reused' ? 'Using existing' : 'Started';
|
const verb = result.status === 'reused' ? 'Using existing' : 'Started';
|
||||||
io.stdout.write(`${verb} KTX daemon\n`);
|
io.stdout.write(`${verb} ktx daemon\n`);
|
||||||
io.stdout.write(`url: ${result.baseUrl}\n`);
|
io.stdout.write(`url: ${result.baseUrl}\n`);
|
||||||
io.stdout.write(`pid: ${result.state.pid}\n`);
|
io.stdout.write(`pid: ${result.state.pid}\n`);
|
||||||
io.stdout.write(`version: ${result.state.version}\n`);
|
io.stdout.write(`version: ${result.state.version}\n`);
|
||||||
|
|
@ -68,10 +68,10 @@ function writeDaemonStart(io: KtxCliIo, result: ManagedPythonDaemonStartResult):
|
||||||
|
|
||||||
function writeDaemonStop(io: KtxCliIo, result: ManagedPythonDaemonStopResult): void {
|
function writeDaemonStop(io: KtxCliIo, result: ManagedPythonDaemonStopResult): void {
|
||||||
if (result.status === 'already-stopped') {
|
if (result.status === 'already-stopped') {
|
||||||
io.stdout.write('KTX daemon already stopped\n');
|
io.stdout.write('ktx daemon already stopped\n');
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
io.stdout.write('Stopped KTX daemon\n');
|
io.stdout.write('Stopped ktx daemon\n');
|
||||||
io.stdout.write(`pid: ${result.state?.pid ?? 'unknown'}\n`);
|
io.stdout.write(`pid: ${result.state?.pid ?? 'unknown'}\n`);
|
||||||
io.stdout.write(`state: ${result.layout.daemonStatePath}\n`);
|
io.stdout.write(`state: ${result.layout.daemonStatePath}\n`);
|
||||||
}
|
}
|
||||||
|
|
@ -94,11 +94,11 @@ function writeDaemonStopAll(io: KtxCliIo, result: ManagedPythonDaemonStopAllResu
|
||||||
result.failed.length === 0 &&
|
result.failed.length === 0 &&
|
||||||
result.scanErrors.length === 0
|
result.scanErrors.length === 0
|
||||||
) {
|
) {
|
||||||
io.stdout.write('No KTX daemons found\n');
|
io.stdout.write('No ktx daemons found\n');
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
if (failed === 0) {
|
if (failed === 0) {
|
||||||
io.stdout.write(`Stopped ${result.stopped.length} KTX daemons\n`);
|
io.stdout.write(`Stopped ${result.stopped.length} ktx daemons\n`);
|
||||||
if (result.stale.length > 0) {
|
if (result.stale.length > 0) {
|
||||||
io.stdout.write(`Cleaned ${result.stale.length} stale daemon states\n`);
|
io.stdout.write(`Cleaned ${result.stale.length} stale daemon states\n`);
|
||||||
}
|
}
|
||||||
|
|
@ -111,7 +111,7 @@ function writeDaemonStopAll(io: KtxCliIo, result: ManagedPythonDaemonStopAllResu
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
io.stderr.write(
|
io.stderr.write(
|
||||||
`Stopped ${result.stopped.length} KTX daemons; failed ${result.failed.length}${
|
`Stopped ${result.stopped.length} ktx daemons; failed ${result.failed.length}${
|
||||||
result.stale.length > 0 ? `; cleaned stale ${result.stale.length}` : ''
|
result.stale.length > 0 ? `; cleaned stale ${result.stale.length}` : ''
|
||||||
}\n`,
|
}\n`,
|
||||||
);
|
);
|
||||||
|
|
@ -129,7 +129,7 @@ function writeDaemonStopAll(io: KtxCliIo, result: ManagedPythonDaemonStopAllResu
|
||||||
}
|
}
|
||||||
|
|
||||||
function writeStatus(io: KtxCliIo, status: ManagedPythonRuntimeStatus): void {
|
function writeStatus(io: KtxCliIo, status: ManagedPythonRuntimeStatus): void {
|
||||||
io.stdout.write('KTX Python runtime\n');
|
io.stdout.write('ktx Python runtime\n');
|
||||||
io.stdout.write(`status: ${status.kind}\n`);
|
io.stdout.write(`status: ${status.kind}\n`);
|
||||||
io.stdout.write(`detail: ${status.detail}\n`);
|
io.stdout.write(`detail: ${status.detail}\n`);
|
||||||
io.stdout.write(`runtime root: ${status.layout.runtimeRoot}\n`);
|
io.stdout.write(`runtime root: ${status.layout.runtimeRoot}\n`);
|
||||||
|
|
@ -142,7 +142,7 @@ function writeStatus(io: KtxCliIo, status: ManagedPythonRuntimeStatus): void {
|
||||||
}
|
}
|
||||||
|
|
||||||
function writeRuntimeChecks(io: KtxCliIo, checks: ManagedPythonRuntimeDoctorCheck[]): void {
|
function writeRuntimeChecks(io: KtxCliIo, checks: ManagedPythonRuntimeDoctorCheck[]): void {
|
||||||
io.stdout.write('KTX Python runtime checks\n');
|
io.stdout.write('ktx Python runtime checks\n');
|
||||||
for (const check of checks) {
|
for (const check of checks) {
|
||||||
io.stdout.write(`${check.status.toUpperCase()} ${check.label}: ${check.detail}\n`);
|
io.stdout.write(`${check.status.toUpperCase()} ${check.label}: ${check.detail}\n`);
|
||||||
if (check.fix) {
|
if (check.fix) {
|
||||||
|
|
|
||||||
|
|
@ -259,7 +259,7 @@ function writeHumanReportBody(report: KtxScanReport, io: KtxCliIo): void {
|
||||||
|
|
||||||
function writeRunSummary(report: KtxScanReport, projectDir: string, io: KtxCliIo): void {
|
function writeRunSummary(report: KtxScanReport, projectDir: string, io: KtxCliIo): void {
|
||||||
const styled = shouldUseStyledOutput(io);
|
const styled = shouldUseStyledOutput(io);
|
||||||
io.stdout.write(`${styled ? green('✓') : ''}${styled ? ' ' : ''}KTX scan completed\n`);
|
io.stdout.write(`${styled ? green('✓') : ''}${styled ? ' ' : ''}ktx scan completed\n`);
|
||||||
io.stdout.write('Status: done\n');
|
io.stdout.write('Status: done\n');
|
||||||
writeHumanReportBody(report, io);
|
writeHumanReportBody(report, io);
|
||||||
const projectDirArg = quoteCliArg(projectDir);
|
const projectDirArg = quoteCliArg(projectDir);
|
||||||
|
|
|
||||||
|
|
@ -234,7 +234,7 @@ function codexSnippet(endpoint: KtxMcpEndpointInfo): string {
|
||||||
if (endpoint.tokenAuth) {
|
if (endpoint.tokenAuth) {
|
||||||
return [
|
return [
|
||||||
'Codex MCP config does not currently document HTTP headers.',
|
'Codex MCP config does not currently document HTTP headers.',
|
||||||
'Run KTX on loopback without token auth for Codex, or configure headers after Codex documents support.',
|
'Run ktx on loopback without token auth for Codex, or configure headers after Codex documents support.',
|
||||||
].join('\n');
|
].join('\n');
|
||||||
}
|
}
|
||||||
return [`[mcp_servers.ktx]`, `url = "${endpoint.url}"`].join('\n');
|
return [`[mcp_servers.ktx]`, `url = "${endpoint.url}"`].join('\n');
|
||||||
|
|
@ -538,16 +538,16 @@ function cliInstructionContent(input: { projectDir: string; launcher: KtxCliLaun
|
||||||
return [
|
return [
|
||||||
'---',
|
'---',
|
||||||
'name: ktx',
|
'name: ktx',
|
||||||
'description: Use local KTX semantic context and wiki knowledge for this project.',
|
'description: Use local ktx semantic context and wiki knowledge for this project.',
|
||||||
'---',
|
'---',
|
||||||
'',
|
'',
|
||||||
'# KTX Local Context',
|
'# ktx Local Context',
|
||||||
'',
|
'',
|
||||||
'This is an admin/developer CLI helper. End-user data agents should use the KTX MCP tools when available.',
|
'This is an admin/developer CLI helper. End-user data agents should use the ktx MCP tools when available.',
|
||||||
'',
|
'',
|
||||||
`Use this project with \`--project-dir ${input.projectDir}\`.`,
|
`Use this project with \`--project-dir ${input.projectDir}\`.`,
|
||||||
'Commands are pinned to the local KTX CLI path that created this file, so agents do not need `ktx` in PATH.',
|
'Commands are pinned to the local ktx CLI path that created this file, so agents do not need `ktx` in PATH.',
|
||||||
'If the CLI path no longer exists after moving this checkout or reinstalling KTX, rerun `ktx setup --agents`.',
|
'If the CLI path no longer exists after moving this checkout or reinstalling ktx, rerun `ktx setup --agents`.',
|
||||||
'',
|
'',
|
||||||
'Agents must not print secrets, credential references, environment variable values, or file contents from ' +
|
'Agents must not print secrets, credential references, environment variable values, or file contents from ' +
|
||||||
'`.ktx/secrets`.',
|
'`.ktx/secrets`.',
|
||||||
|
|
@ -676,7 +676,7 @@ function mergeManifest(
|
||||||
export async function removeKtxAgentInstall(projectDir: string, io: KtxCliIo): Promise<number> {
|
export async function removeKtxAgentInstall(projectDir: string, io: KtxCliIo): Promise<number> {
|
||||||
const manifest = await readKtxAgentInstallManifest(projectDir);
|
const manifest = await readKtxAgentInstallManifest(projectDir);
|
||||||
if (!manifest) {
|
if (!manifest) {
|
||||||
io.stdout.write('No KTX agent installation manifest found.\n');
|
io.stdout.write('No ktx agent installation manifest found.\n');
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
for (const entry of manifest.entries) {
|
for (const entry of manifest.entries) {
|
||||||
|
|
@ -684,7 +684,7 @@ export async function removeKtxAgentInstall(projectDir: string, io: KtxCliIo): P
|
||||||
if (entry.kind === 'json-key') await removeJsonKey(entry.path, entry.jsonPath).catch(() => undefined);
|
if (entry.kind === 'json-key') await removeJsonKey(entry.path, entry.jsonPath).catch(() => undefined);
|
||||||
}
|
}
|
||||||
await rm(agentInstallManifestPath(projectDir), { force: true });
|
await rm(agentInstallManifestPath(projectDir), { force: true });
|
||||||
io.stdout.write('Removed KTX agent integration files from manifest.\n');
|
io.stdout.write('Removed ktx agent integration files from manifest.\n');
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -990,7 +990,7 @@ function formatAgentNextActions(input: {
|
||||||
if (claudeCodeInstall) {
|
if (claudeCodeInstall) {
|
||||||
lines.push(`${step}. Open Claude Code`);
|
lines.push(`${step}. Open Claude Code`);
|
||||||
if (claudeCodeInstall.scope === 'project') {
|
if (claudeCodeInstall.scope === 'project') {
|
||||||
lines.push(' Open Claude Code from the KTX project directory:');
|
lines.push(' Open Claude Code from the ktx project directory:');
|
||||||
lines.push('');
|
lines.push('');
|
||||||
lines.push(' RUN:');
|
lines.push(' RUN:');
|
||||||
lines.push(` cd ${shellScriptQuote(projectDir)}`);
|
lines.push(` cd ${shellScriptQuote(projectDir)}`);
|
||||||
|
|
@ -1007,7 +1007,7 @@ function formatAgentNextActions(input: {
|
||||||
if (cursorInstall) {
|
if (cursorInstall) {
|
||||||
lines.push(`${step}. Open Cursor`);
|
lines.push(`${step}. Open Cursor`);
|
||||||
if (cursorInstall.scope === 'project') {
|
if (cursorInstall.scope === 'project') {
|
||||||
lines.push(' Open Cursor from the KTX project directory:');
|
lines.push(' Open Cursor from the ktx project directory:');
|
||||||
lines.push('');
|
lines.push('');
|
||||||
lines.push(' OPEN:');
|
lines.push(' OPEN:');
|
||||||
lines.push(` ${projectDir}`);
|
lines.push(` ${projectDir}`);
|
||||||
|
|
@ -1020,7 +1020,7 @@ function formatAgentNextActions(input: {
|
||||||
|
|
||||||
if (input.installs.some((install) => install.target === 'claude-desktop')) {
|
if (input.installs.some((install) => install.target === 'claude-desktop')) {
|
||||||
lines.push(`${step}. Restart Claude Desktop`);
|
lines.push(`${step}. Restart Claude Desktop`);
|
||||||
lines.push(' Claude Desktop loads KTX MCP after restart.');
|
lines.push(' Claude Desktop loads ktx MCP after restart.');
|
||||||
pushBlankLine(lines);
|
pushBlankLine(lines);
|
||||||
step += 1;
|
step += 1;
|
||||||
|
|
||||||
|
|
@ -1032,7 +1032,7 @@ function formatAgentNextActions(input: {
|
||||||
for (const path of skillBundlePaths) {
|
for (const path of skillBundlePaths) {
|
||||||
lines.push(` ${path}`);
|
lines.push(` ${path}`);
|
||||||
}
|
}
|
||||||
lines.push(' Toggle the uploaded KTX skills on.');
|
lines.push(' Toggle the uploaded ktx skills on.');
|
||||||
pushBlankLine(lines);
|
pushBlankLine(lines);
|
||||||
step += 1;
|
step += 1;
|
||||||
}
|
}
|
||||||
|
|
@ -1104,16 +1104,16 @@ export async function runKtxSetupAgentsStep(
|
||||||
args.inputMode === 'disabled'
|
args.inputMode === 'disabled'
|
||||||
? args.mode
|
? args.mode
|
||||||
: ((await prompts.select({
|
: ((await prompts.select({
|
||||||
message: 'What should agents be allowed to do with this KTX project?',
|
message: 'What should agents be allowed to do with this ktx project?',
|
||||||
options: [
|
options: [
|
||||||
{
|
{
|
||||||
value: 'mcp',
|
value: 'mcp',
|
||||||
label: 'Ask data questions with KTX MCP',
|
label: 'Ask data questions with ktx MCP',
|
||||||
hint: 'Installs the MCP connection and analytics workflow skill. Best for normal use.',
|
hint: 'Installs the MCP connection and analytics workflow skill. Best for normal use.',
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
value: 'mcp-cli',
|
value: 'mcp-cli',
|
||||||
label: 'Ask data questions + manage KTX with CLI commands',
|
label: 'Ask data questions + manage ktx with CLI commands',
|
||||||
hint: 'Adds an admin CLI skill so agents can run ktx status, sl, wiki, and setup commands.',
|
hint: 'Adds an admin CLI skill so agents can run ktx status, sl, wiki, and setup commands.',
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|
@ -1135,7 +1135,7 @@ export async function runKtxSetupAgentsStep(
|
||||||
: args.inputMode === 'disabled'
|
: args.inputMode === 'disabled'
|
||||||
? []
|
? []
|
||||||
: ((await prompts.multiselect({
|
: ((await prompts.multiselect({
|
||||||
message: 'Which agent targets should KTX install?',
|
message: 'Which agent targets should ktx install?',
|
||||||
options: [
|
options: [
|
||||||
{ value: 'claude-code', label: 'Claude Code' },
|
{ value: 'claude-code', label: 'Claude Code' },
|
||||||
{ value: 'claude-desktop', label: 'Claude Desktop' },
|
{ value: 'claude-desktop', label: 'Claude Desktop' },
|
||||||
|
|
@ -1163,17 +1163,17 @@ export async function runKtxSetupAgentsStep(
|
||||||
scopeTargets.length > 0 &&
|
scopeTargets.length > 0 &&
|
||||||
scopeTargets.every(targetSupportsGlobalScope)
|
scopeTargets.every(targetSupportsGlobalScope)
|
||||||
? ((await prompts.select({
|
? ((await prompts.select({
|
||||||
message: `Where should KTX install supported agent config?\n\nKTX project: ${resolve(args.projectDir)}`,
|
message: `Where should ktx install supported agent config?\n\nktx project: ${resolve(args.projectDir)}`,
|
||||||
options: [
|
options: [
|
||||||
{
|
{
|
||||||
value: 'project',
|
value: 'project',
|
||||||
label: 'Project scope (KTX project directory)',
|
label: 'Project scope (ktx project directory)',
|
||||||
hint: 'Only agents opened from this KTX project path load the project-scoped config.',
|
hint: 'Only agents opened from this ktx project path load the project-scoped config.',
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
value: 'global',
|
value: 'global',
|
||||||
label: 'Global scope (user config)',
|
label: 'Global scope (user config)',
|
||||||
hint: 'Agents can load this KTX project from any working directory.',
|
hint: 'Agents can load this ktx project from any working directory.',
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
})) as KtxAgentScope | 'back')
|
})) as KtxAgentScope | 'back')
|
||||||
|
|
|
||||||
|
|
@ -345,7 +345,7 @@ async function prepareBuildTargets(args: KtxSetupContextStepArgs, io: KtxCliIo):
|
||||||
if (args.allowEmpty === true) {
|
if (args.allowEmpty === true) {
|
||||||
return { kind: 'result', result: { status: 'skipped', projectDir: args.projectDir } };
|
return { kind: 'result', result: { status: 'skipped', projectDir: args.projectDir } };
|
||||||
}
|
}
|
||||||
io.stderr.write('No databases or context sources are configured for a KTX context build.\n');
|
io.stderr.write('No databases or context sources are configured for a ktx context build.\n');
|
||||||
return { kind: 'result', result: { status: 'failed', projectDir: args.projectDir } };
|
return { kind: 'result', result: { status: 'failed', projectDir: args.projectDir } };
|
||||||
}
|
}
|
||||||
const preflightPlan = buildPublicIngestPlan(project, { projectDir: project.projectDir, all: true });
|
const preflightPlan = buildPublicIngestPlan(project, { projectDir: project.projectDir, all: true });
|
||||||
|
|
@ -367,12 +367,12 @@ function writeConnectionGateFailureLines(
|
||||||
projectDir: string,
|
projectDir: string,
|
||||||
failures: ConnectionGateFailure[],
|
failures: ConnectionGateFailure[],
|
||||||
): void {
|
): void {
|
||||||
io.stderr.write('KTX cannot build context: a required connection failed its live test.\n\n');
|
io.stderr.write('ktx cannot build context: a required connection failed its live test.\n\n');
|
||||||
io.stderr.write('Failed connections:\n');
|
io.stderr.write('Failed connections:\n');
|
||||||
for (const failure of failures) {
|
for (const failure of failures) {
|
||||||
io.stderr.write(` ${failure.connectionId} (${failure.driver})\n`);
|
io.stderr.write(` ${failure.connectionId} (${failure.driver})\n`);
|
||||||
}
|
}
|
||||||
io.stderr.write('\nEach connection must be reachable before KTX builds context.\n');
|
io.stderr.write('\nEach connection must be reachable before ktx builds context.\n');
|
||||||
io.stderr.write(
|
io.stderr.write(
|
||||||
`Run \`ktx connection test <id> --project-dir ${resolve(projectDir)}\` to see the error, fix the connection, then retry.\n`,
|
`Run \`ktx connection test <id> --project-dir ${resolve(projectDir)}\` to see the error, fix the connection, then retry.\n`,
|
||||||
);
|
);
|
||||||
|
|
@ -570,7 +570,7 @@ async function markContextComplete(projectDir: string): Promise<void> {
|
||||||
}
|
}
|
||||||
|
|
||||||
function writeMissingCapabilities(missing: string[], io: KtxCliIo): void {
|
function writeMissingCapabilities(missing: string[], io: KtxCliIo): void {
|
||||||
io.stderr.write('KTX cannot build agent-ready context yet.\n\n');
|
io.stderr.write('ktx cannot build agent-ready context yet.\n\n');
|
||||||
io.stderr.write('Missing:\n');
|
io.stderr.write('Missing:\n');
|
||||||
for (const item of missing) {
|
for (const item of missing) {
|
||||||
io.stderr.write(` ${item}\n`);
|
io.stderr.write(` ${item}\n`);
|
||||||
|
|
@ -589,7 +589,7 @@ function writeSuccess(
|
||||||
targets: KtxSetupContextTargets,
|
targets: KtxSetupContextTargets,
|
||||||
io: KtxCliIo,
|
io: KtxCliIo,
|
||||||
): void {
|
): void {
|
||||||
io.stdout.write('\nKTX context is ready for agents.\n\n');
|
io.stdout.write('\nktx context is ready for agents.\n\n');
|
||||||
io.stdout.write('Databases:\n');
|
io.stdout.write('Databases:\n');
|
||||||
if (targets.primarySourceConnectionIds.length === 0) {
|
if (targets.primarySourceConnectionIds.length === 0) {
|
||||||
io.stdout.write(' none\n');
|
io.stdout.write(' none\n');
|
||||||
|
|
@ -612,7 +612,7 @@ function writeSuccess(
|
||||||
}
|
}
|
||||||
|
|
||||||
function writeExistingContextSuccess(readiness: KtxSetupContextReadiness, io: KtxCliIo): void {
|
function writeExistingContextSuccess(readiness: KtxSetupContextReadiness, io: KtxCliIo): void {
|
||||||
io.stdout.write('\nKTX context is ready for agents.\n\n');
|
io.stdout.write('\nktx context is ready for agents.\n\n');
|
||||||
io.stdout.write('Existing context artifacts were found from setup ingest.\n\n');
|
io.stdout.write('Existing context artifacts were found from setup ingest.\n\n');
|
||||||
io.stdout.write('Verification:\n');
|
io.stdout.write('Verification:\n');
|
||||||
io.stdout.write(` Agent context: ${readiness.agentContextReady ? 'ready' : 'not ready'}\n`);
|
io.stdout.write(` Agent context: ${readiness.agentContextReady ? 'ready' : 'not ready'}\n`);
|
||||||
|
|
@ -622,8 +622,8 @@ function writeExistingContextSuccess(readiness: KtxSetupContextReadiness, io: Kt
|
||||||
async function promptForBuild(prompts: KtxSetupContextPromptAdapter): Promise<'build' | 'skip' | 'back'> {
|
async function promptForBuild(prompts: KtxSetupContextPromptAdapter): Promise<'build' | 'skip' | 'back'> {
|
||||||
return (await prompts.select({
|
return (await prompts.select({
|
||||||
message:
|
message:
|
||||||
'Build KTX context for agents?\n\n' +
|
'Build ktx context for agents?\n\n' +
|
||||||
'KTX is fully configured and ready to build context. This may take a few minutes to a few hours.',
|
'ktx is fully configured and ready to build context. This may take a few minutes to a few hours.',
|
||||||
options: [
|
options: [
|
||||||
{ value: 'build', label: 'Build context now (recommended)' },
|
{ value: 'build', label: 'Build context now (recommended)' },
|
||||||
{ value: 'skip', label: 'Leave context unbuilt and exit setup' },
|
{ value: 'skip', label: 'Leave context unbuilt and exit setup' },
|
||||||
|
|
@ -716,7 +716,7 @@ async function runBuild(
|
||||||
failureReason: readiness.details.join(' '),
|
failureReason: readiness.details.join(' '),
|
||||||
...(lastSourceProgress ? { sourceProgress: lastSourceProgress } : {}),
|
...(lastSourceProgress ? { sourceProgress: lastSourceProgress } : {}),
|
||||||
});
|
});
|
||||||
io.stderr.write('KTX context build did not pass agent-readiness verification.\n');
|
io.stderr.write('ktx context build did not pass agent-readiness verification.\n');
|
||||||
for (const detail of readiness.details) {
|
for (const detail of readiness.details) {
|
||||||
io.stderr.write(` ${detail}\n`);
|
io.stderr.write(` ${detail}\n`);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -270,7 +270,7 @@ function driverLabel(driver: KtxSetupDatabaseDriver): string {
|
||||||
}
|
}
|
||||||
|
|
||||||
function connectionNamePrompt(label: string): string {
|
function connectionNamePrompt(label: string): string {
|
||||||
return `Name this ${label} connection\nKTX will use this short name in commands and config. You can rename it now.`;
|
return `Name this ${label} connection\nktx will use this short name in commands and config. You can rename it now.`;
|
||||||
}
|
}
|
||||||
|
|
||||||
function missingConnectionDetailsPrompt(
|
function missingConnectionDetailsPrompt(
|
||||||
|
|
@ -324,13 +324,6 @@ function numberConfigField(connection: KtxProjectConnectionConfig | undefined, f
|
||||||
return typeof value === 'number' && Number.isFinite(value) ? value : undefined;
|
return typeof value === 'number' && Number.isFinite(value) ? value : undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
function historicSqlConfigRecord(connection: KtxProjectConnectionConfig | undefined): Record<string, unknown> | null {
|
|
||||||
const historicSql = connection?.historicSql;
|
|
||||||
return historicSql && typeof historicSql === 'object' && !Array.isArray(historicSql)
|
|
||||||
? (historicSql as Record<string, unknown>)
|
|
||||||
: null;
|
|
||||||
}
|
|
||||||
|
|
||||||
function contextRecord(connection: KtxProjectConnectionConfig | undefined): Record<string, unknown> {
|
function contextRecord(connection: KtxProjectConnectionConfig | undefined): Record<string, unknown> {
|
||||||
const context = connection?.context;
|
const context = connection?.context;
|
||||||
return context && typeof context === 'object' && !Array.isArray(context) ? (context as Record<string, unknown>) : {};
|
return context && typeof context === 'object' && !Array.isArray(context) ? (context as Record<string, unknown>) : {};
|
||||||
|
|
@ -343,19 +336,12 @@ function queryHistoryConfigRecord(connection: KtxProjectConnectionConfig | undef
|
||||||
: null;
|
: null;
|
||||||
}
|
}
|
||||||
|
|
||||||
function stripLegacyHistoricSql(connection: KtxProjectConnectionConfig): KtxProjectConnectionConfig {
|
|
||||||
const { historicSql: _historicSql, ...rest } = connection as KtxProjectConnectionConfig & {
|
|
||||||
historicSql?: unknown;
|
|
||||||
};
|
|
||||||
return rest;
|
|
||||||
}
|
|
||||||
|
|
||||||
function withQueryHistoryConfig(
|
function withQueryHistoryConfig(
|
||||||
connection: KtxProjectConnectionConfig,
|
connection: KtxProjectConnectionConfig,
|
||||||
queryHistory: Record<string, unknown>,
|
queryHistory: Record<string, unknown>,
|
||||||
): KtxProjectConnectionConfig {
|
): KtxProjectConnectionConfig {
|
||||||
return {
|
return {
|
||||||
...stripLegacyHistoricSql(connection),
|
...connection,
|
||||||
context: {
|
context: {
|
||||||
...contextRecord(connection),
|
...contextRecord(connection),
|
||||||
queryHistory,
|
queryHistory,
|
||||||
|
|
@ -363,16 +349,6 @@ function withQueryHistoryConfig(
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
function migrateLegacyHistoricSqlConnection(connection: KtxProjectConnectionConfig): KtxProjectConnectionConfig {
|
|
||||||
const existingQueryHistory = queryHistoryConfigRecord(connection);
|
|
||||||
const legacy = historicSqlConfigRecord(connection);
|
|
||||||
if (existingQueryHistory || !legacy) {
|
|
||||||
return existingQueryHistory ? stripLegacyHistoricSql(connection) : connection;
|
|
||||||
}
|
|
||||||
const { dialect: _dialect, ...queryHistory } = legacy;
|
|
||||||
return withQueryHistoryConfig(connection, queryHistory);
|
|
||||||
}
|
|
||||||
|
|
||||||
function setupHistoricSqlProbeResult(
|
function setupHistoricSqlProbeResult(
|
||||||
outcome: HistoricSqlProbeOutcome | null,
|
outcome: HistoricSqlProbeOutcome | null,
|
||||||
): KtxSetupHistoricSqlProbeResult {
|
): KtxSetupHistoricSqlProbeResult {
|
||||||
|
|
@ -1203,7 +1179,7 @@ async function disableConnectionQueryHistory(projectDir: string, connectionId: s
|
||||||
if (!connection) {
|
if (!connection) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const existing = queryHistoryConfigRecord(connection) ?? historicSqlConfigRecord(connection) ?? {};
|
const existing = queryHistoryConfigRecord(connection) ?? {};
|
||||||
await writeConnectionConfig({
|
await writeConnectionConfig({
|
||||||
projectDir,
|
projectDir,
|
||||||
connectionId,
|
connectionId,
|
||||||
|
|
@ -1560,18 +1536,7 @@ async function ensureHistoricSqlIngestDefaults(projectDir: string): Promise<void
|
||||||
|
|
||||||
async function markDatabasesComplete(projectDir: string, connectionIds: string[]): Promise<void> {
|
async function markDatabasesComplete(projectDir: string, connectionIds: string[]): Promise<void> {
|
||||||
const project = await loadKtxProject({ projectDir });
|
const project = await loadKtxProject({ projectDir });
|
||||||
const config = setKtxSetupDatabaseConnectionIds(
|
const config = setKtxSetupDatabaseConnectionIds(project.config, unique(connectionIds));
|
||||||
{
|
|
||||||
...project.config,
|
|
||||||
connections: Object.fromEntries(
|
|
||||||
Object.entries(project.config.connections).map(([connectionId, connection]) => [
|
|
||||||
connectionId,
|
|
||||||
migrateLegacyHistoricSqlConnection(connection),
|
|
||||||
]),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
unique(connectionIds),
|
|
||||||
);
|
|
||||||
await writeFile(project.configPath, serializeKtxProjectConfig(config), 'utf-8');
|
await writeFile(project.configPath, serializeKtxProjectConfig(config), 'utf-8');
|
||||||
await markKtxSetupStateStepComplete(projectDir, 'databases');
|
await markKtxSetupStateStepComplete(projectDir, 'databases');
|
||||||
}
|
}
|
||||||
|
|
@ -1584,7 +1549,7 @@ async function maybeRunHistoricSqlSetupProbe(input: {
|
||||||
}): Promise<boolean> {
|
}): Promise<boolean> {
|
||||||
const project = await loadKtxProject({ projectDir: input.projectDir });
|
const project = await loadKtxProject({ projectDir: input.projectDir });
|
||||||
const connection = project.config.connections[input.connectionId];
|
const connection = project.config.connections[input.connectionId];
|
||||||
const queryHistory = queryHistoryConfigRecord(connection) ?? historicSqlConfigRecord(connection);
|
const queryHistory = queryHistoryConfigRecord(connection);
|
||||||
if (queryHistory?.enabled !== true) {
|
if (queryHistory?.enabled !== true) {
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
@ -1994,7 +1959,7 @@ async function chooseDrivers(
|
||||||
}
|
}
|
||||||
if (args.inputMode === 'disabled') {
|
if (args.inputMode === 'disabled') {
|
||||||
io.stderr.write(
|
io.stderr.write(
|
||||||
'KTX cannot work without a database. Pass --database or --database-connection-id, or pass --skip-databases to leave setup incomplete.\n',
|
'ktx cannot work without a database. Pass --database or --database-connection-id, or pass --skip-databases to leave setup incomplete.\n',
|
||||||
);
|
);
|
||||||
return 'missing-input';
|
return 'missing-input';
|
||||||
}
|
}
|
||||||
|
|
@ -2005,7 +1970,7 @@ async function chooseDrivers(
|
||||||
io,
|
io,
|
||||||
);
|
);
|
||||||
const choices = await prompts.multiselect({
|
const choices = await prompts.multiselect({
|
||||||
message: withMultiselectNavigation('Which databases should KTX connect to?'),
|
message: withMultiselectNavigation('Which databases should ktx connect to?'),
|
||||||
options: [...DRIVER_OPTIONS],
|
options: [...DRIVER_OPTIONS],
|
||||||
...(initialValues.length > 0 ? { initialValues } : {}),
|
...(initialValues.length > 0 ? { initialValues } : {}),
|
||||||
required: true,
|
required: true,
|
||||||
|
|
@ -2285,7 +2250,7 @@ export async function runKtxSetupDatabasesStep(
|
||||||
deps: KtxSetupDatabasesDeps = {},
|
deps: KtxSetupDatabasesDeps = {},
|
||||||
): Promise<KtxSetupDatabasesResult> {
|
): Promise<KtxSetupDatabasesResult> {
|
||||||
if (args.skipDatabases) {
|
if (args.skipDatabases) {
|
||||||
io.stdout.write('│ Database setup skipped. KTX cannot work until you add a database.\n');
|
io.stdout.write('│ Database setup skipped. ktx cannot work until you add a database.\n');
|
||||||
return { status: 'skipped', projectDir: args.projectDir };
|
return { status: 'skipped', projectDir: args.projectDir };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -2404,7 +2369,7 @@ export async function runKtxSetupDatabasesStep(
|
||||||
if (drivers === 'missing-input') return { status: 'missing-input', projectDir: args.projectDir };
|
if (drivers === 'missing-input') return { status: 'missing-input', projectDir: args.projectDir };
|
||||||
if (drivers.length === 0) {
|
if (drivers.length === 0) {
|
||||||
await markDatabasesComplete(args.projectDir, []);
|
await markDatabasesComplete(args.projectDir, []);
|
||||||
io.stdout.write('│ KTX cannot work without a database.\n');
|
io.stdout.write('│ ktx cannot work without a database.\n');
|
||||||
return { status: 'skipped', projectDir: args.projectDir };
|
return { status: 'skipped', projectDir: args.projectDir };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -68,7 +68,7 @@ function createTargetState(target: KtxPublicIngestPlanTarget): ContextBuildTarge
|
||||||
export function renderDemoBanner(projectDir?: string): string {
|
export function renderDemoBanner(projectDir?: string): string {
|
||||||
const lines = [
|
const lines = [
|
||||||
'',
|
'',
|
||||||
`┌ ${cyan('Demo mode')} — data has been pre-processed and KTX context is already built.`,
|
`┌ ${cyan('Demo mode')} — data has been pre-processed and ktx context is already built.`,
|
||||||
'│ This walkthrough illustrates the setup steps. Selections are pre-filled and read-only.',
|
'│ This walkthrough illustrates the setup steps. Selections are pre-filled and read-only.',
|
||||||
];
|
];
|
||||||
if (projectDir) {
|
if (projectDir) {
|
||||||
|
|
@ -95,7 +95,7 @@ export function renderDemoAgentTransition(): string {
|
||||||
const lines = [
|
const lines = [
|
||||||
'┌ Demo project is ready — let\'s connect your agent',
|
'┌ Demo project is ready — let\'s connect your agent',
|
||||||
'│',
|
'│',
|
||||||
'│ Your KTX context has been built with demo data.',
|
'│ Your ktx context has been built with demo data.',
|
||||||
'│ Select an agent to start using it.',
|
'│ Select an agent to start using it.',
|
||||||
'└',
|
'└',
|
||||||
];
|
];
|
||||||
|
|
@ -106,12 +106,12 @@ export function renderDemoAgentTransition(): string {
|
||||||
export function renderDemoCompletionSummary(projectDir: string, agentInstalled: boolean): string {
|
export function renderDemoCompletionSummary(projectDir: string, agentInstalled: boolean): string {
|
||||||
const lines: string[] = [
|
const lines: string[] = [
|
||||||
'',
|
'',
|
||||||
`${cyan('★')} KTX demo is ready`,
|
`${cyan('★')} ktx demo is ready`,
|
||||||
'',
|
'',
|
||||||
];
|
];
|
||||||
|
|
||||||
if (agentInstalled) {
|
if (agentInstalled) {
|
||||||
lines.push(' Your agent is connected to a demo KTX project.');
|
lines.push(' Your agent is connected to a demo ktx project.');
|
||||||
} else {
|
} else {
|
||||||
lines.push(' Demo project created. Connect an agent to start using it:');
|
lines.push(' Demo project created. Connect an agent to start using it:');
|
||||||
lines.push(` $ ${cyan(`ktx setup --agents --project-dir ${projectDir}`)}`);
|
lines.push(` $ ${cyan(`ktx setup --agents --project-dir ${projectDir}`)}`);
|
||||||
|
|
@ -120,7 +120,7 @@ export function renderDemoCompletionSummary(projectDir: string, agentInstalled:
|
||||||
lines.push(
|
lines.push(
|
||||||
'',
|
'',
|
||||||
` ${dim('⚠')} This project is in a temporary directory and will be`,
|
` ${dim('⚠')} This project is in a temporary directory and will be`,
|
||||||
' cleaned up by your system. To set up KTX with your own',
|
' cleaned up by your system. To set up ktx with your own',
|
||||||
' data, run: ktx setup',
|
' data, run: ktx setup',
|
||||||
'',
|
'',
|
||||||
` Project: ${projectDir}`,
|
` Project: ${projectDir}`,
|
||||||
|
|
@ -234,9 +234,9 @@ export function buildDemoReplayTimeline(): DemoReplayEvent[] {
|
||||||
function renderDemoContextCompletionSummary(): string {
|
function renderDemoContextCompletionSummary(): string {
|
||||||
const lines = [
|
const lines = [
|
||||||
'',
|
'',
|
||||||
`${cyan('★')} KTX finished building context`,
|
`${cyan('★')} ktx finished building context`,
|
||||||
'',
|
'',
|
||||||
' KTX created:',
|
' ktx created:',
|
||||||
` ${cyan('📊')} 46 semantic layer definitions`,
|
` ${cyan('📊')} 46 semantic layer definitions`,
|
||||||
` ${cyan('📝')} 28 wiki pages`,
|
` ${cyan('📝')} 28 wiki pages`,
|
||||||
'',
|
'',
|
||||||
|
|
|
||||||
|
|
@ -80,7 +80,7 @@ const DEFAULTS: Record<
|
||||||
|
|
||||||
const LOCAL_EMBEDDING_BACKEND: KtxSetupEmbeddingBackend = 'sentence-transformers';
|
const LOCAL_EMBEDDING_BACKEND: KtxSetupEmbeddingBackend = 'sentence-transformers';
|
||||||
const EMBEDDING_OPTION_PROMPT_CONTEXT =
|
const EMBEDDING_OPTION_PROMPT_CONTEXT =
|
||||||
'KTX uses embeddings for semantic search over semantic-layer sources, wiki context, schema metadata, ' +
|
'ktx uses embeddings for semantic search over semantic-layer sources, wiki context, schema metadata, ' +
|
||||||
'and relationship evidence.';
|
'and relationship evidence.';
|
||||||
const LOCAL_EMBEDDING_HEALTH_TIMEOUT_MS = 120_000;
|
const LOCAL_EMBEDDING_HEALTH_TIMEOUT_MS = 120_000;
|
||||||
const LOCAL_EMBEDDING_STDERR_TAIL_LINES = 40;
|
const LOCAL_EMBEDDING_STDERR_TAIL_LINES = 40;
|
||||||
|
|
@ -220,7 +220,7 @@ async function chooseCredentialRef(
|
||||||
const defaultEnv = DEFAULTS[backend].envName ?? 'EMBEDDING_API_KEY';
|
const defaultEnv = DEFAULTS[backend].envName ?? 'EMBEDDING_API_KEY';
|
||||||
const prompts = deps.prompts ?? createPromptAdapter();
|
const prompts = deps.prompts ?? createPromptAdapter();
|
||||||
const choice = await prompts.select({
|
const choice = await prompts.select({
|
||||||
message: `How should KTX find your ${embeddingBackendDisplayName(backend)} embedding API key?`,
|
message: `How should ktx find your ${embeddingBackendDisplayName(backend)} embedding API key?`,
|
||||||
options: [
|
options: [
|
||||||
{ value: 'paste', label: 'Paste a key and save it as a local secret file' },
|
{ value: 'paste', label: 'Paste a key and save it as a local secret file' },
|
||||||
{ value: 'env', label: `Use ${defaultEnv} from the environment` },
|
{ value: 'env', label: `Use ${defaultEnv} from the environment` },
|
||||||
|
|
@ -233,7 +233,7 @@ async function chooseCredentialRef(
|
||||||
if (choice === 'paste') {
|
if (choice === 'paste') {
|
||||||
io.stdout.write(
|
io.stdout.write(
|
||||||
`│ ${[
|
`│ ${[
|
||||||
`KTX will save the key in .ktx/secrets/${backend}-api-key with local file permissions,`,
|
`ktx will save the key in .ktx/secrets/${backend}-api-key with local file permissions,`,
|
||||||
'then write a file: reference in ktx.yaml.',
|
'then write a file: reference in ktx.yaml.',
|
||||||
].join(' ')}\n`,
|
].join(' ')}\n`,
|
||||||
);
|
);
|
||||||
|
|
@ -272,7 +272,7 @@ async function chooseEmbeddingBackend(
|
||||||
return LOCAL_EMBEDDING_BACKEND;
|
return LOCAL_EMBEDDING_BACKEND;
|
||||||
}
|
}
|
||||||
const choice = await (deps.prompts ?? createPromptAdapter()).select({
|
const choice = await (deps.prompts ?? createPromptAdapter()).select({
|
||||||
message: `Which embedding option should KTX use?\n\n${EMBEDDING_OPTION_PROMPT_CONTEXT}`,
|
message: `Which embedding option should ktx use?\n\n${EMBEDDING_OPTION_PROMPT_CONTEXT}`,
|
||||||
options: [
|
options: [
|
||||||
{ value: 'sentence-transformers', label: 'Local sentence-transformers embeddings' },
|
{ value: 'sentence-transformers', label: 'Local sentence-transformers embeddings' },
|
||||||
{ value: 'openai', label: 'OpenAI embeddings', hint: 'recommended' },
|
{ value: 'openai', label: 'OpenAI embeddings', hint: 'recommended' },
|
||||||
|
|
@ -303,13 +303,13 @@ async function readLocalEmbeddingDaemonStderrTail(stderrLog: string | undefined)
|
||||||
function localEmbeddingSetupMessage(message: string, stderrTail: string[] = []): string {
|
function localEmbeddingSetupMessage(message: string, stderrTail: string[] = []): string {
|
||||||
const lines = [
|
const lines = [
|
||||||
`Local embedding health check failed: ${message}`,
|
`Local embedding health check failed: ${message}`,
|
||||||
'Local embeddings use the KTX-managed Python runtime.',
|
'Local embeddings use the ktx-managed Python runtime.',
|
||||||
'Prepare the runtime with: ktx admin runtime start --feature local-embeddings',
|
'Prepare the runtime with: ktx admin runtime start --feature local-embeddings',
|
||||||
'Use --yes with setup to install and start the runtime without prompting.',
|
'Use --yes with setup to install and start the runtime without prompting.',
|
||||||
'The first run may download Python packages and the all-MiniLM-L6-v2 model.',
|
'The first run may download Python packages and the all-MiniLM-L6-v2 model.',
|
||||||
];
|
];
|
||||||
if (stderrTail.length > 0) {
|
if (stderrTail.length > 0) {
|
||||||
lines.push('Recent KTX daemon stderr:', ...stderrTail);
|
lines.push('Recent ktx daemon stderr:', ...stderrTail);
|
||||||
}
|
}
|
||||||
return lines.join('\n');
|
return lines.join('\n');
|
||||||
}
|
}
|
||||||
|
|
@ -318,7 +318,7 @@ async function promptAfterLocalEmbeddingFailure(
|
||||||
deps: KtxSetupEmbeddingsDeps,
|
deps: KtxSetupEmbeddingsDeps,
|
||||||
): Promise<'retry' | Extract<KtxSetupEmbeddingBackend, 'openai'> | 'back'> {
|
): Promise<'retry' | Extract<KtxSetupEmbeddingBackend, 'openai'> | 'back'> {
|
||||||
const choice = await (deps.prompts ?? createPromptAdapter()).select({
|
const choice = await (deps.prompts ?? createPromptAdapter()).select({
|
||||||
message: 'Local embeddings are not reachable. Start the local KTX daemon, then retry.',
|
message: 'Local embeddings are not reachable. Start the local ktx daemon, then retry.',
|
||||||
options: [
|
options: [
|
||||||
{ value: 'retry', label: 'Retry' },
|
{ value: 'retry', label: 'Retry' },
|
||||||
{ value: 'openai', label: 'Use OpenAI embeddings' },
|
{ value: 'openai', label: 'Use OpenAI embeddings' },
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@ import { cancel, confirm, isCancel as isClackCancel } from '@clack/prompts';
|
||||||
|
|
||||||
export class KtxSetupExitError extends Error {
|
export class KtxSetupExitError extends Error {
|
||||||
constructor() {
|
constructor() {
|
||||||
super('KTX setup exit requested');
|
super('ktx setup exit requested');
|
||||||
this.name = 'KtxSetupExitError';
|
this.name = 'KtxSetupExitError';
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -102,12 +102,12 @@ export interface KtxSetupModelDeps {
|
||||||
}
|
}
|
||||||
|
|
||||||
const ANTHROPIC_CREDENTIAL_PROMPT_CONTEXT =
|
const ANTHROPIC_CREDENTIAL_PROMPT_CONTEXT =
|
||||||
'KTX uses the key to verify Anthropic model access now and to run ingest agents that turn schemas, SQL, ' +
|
'ktx uses the key to verify Anthropic model access now and to run ingest agents that turn schemas, SQL, ' +
|
||||||
'BI metadata, and docs into semantic-layer sources and wiki context. ktx.yaml stores an env: or file: ' +
|
'BI metadata, and docs into semantic-layer sources and wiki context. ktx.yaml stores an env: or file: ' +
|
||||||
'reference, not the raw key.';
|
'reference, not the raw key.';
|
||||||
|
|
||||||
const VERTEX_PROJECT_PROMPT_CONTEXT =
|
const VERTEX_PROJECT_PROMPT_CONTEXT =
|
||||||
'KTX stores the selected Google Cloud project ID in ktx.yaml and uses Application Default Credentials for ' +
|
'ktx stores the selected Google Cloud project ID in ktx.yaml and uses Application Default Credentials for ' +
|
||||||
'access. Project visibility depends on the signed-in Google account and organization permissions.';
|
'access. Project visibility depends on the signed-in Google account and organization permissions.';
|
||||||
const DEFAULT_VERTEX_LOCATION = 'us-east5';
|
const DEFAULT_VERTEX_LOCATION = 'us-east5';
|
||||||
|
|
||||||
|
|
@ -415,7 +415,7 @@ async function chooseCredentialRef(
|
||||||
}
|
}
|
||||||
while (true) {
|
while (true) {
|
||||||
const choice = await prompts.select({
|
const choice = await prompts.select({
|
||||||
message: `How should KTX find your Anthropic API key?\n\n${ANTHROPIC_CREDENTIAL_PROMPT_CONTEXT}`,
|
message: `How should ktx find your Anthropic API key?\n\n${ANTHROPIC_CREDENTIAL_PROMPT_CONTEXT}`,
|
||||||
options: [
|
options: [
|
||||||
{ value: 'paste', label: 'Paste a key and save it as a local secret file' },
|
{ value: 'paste', label: 'Paste a key and save it as a local secret file' },
|
||||||
{ value: 'env', label: 'Use ANTHROPIC_API_KEY from the environment' },
|
{ value: 'env', label: 'Use ANTHROPIC_API_KEY from the environment' },
|
||||||
|
|
@ -427,7 +427,7 @@ async function chooseCredentialRef(
|
||||||
}
|
}
|
||||||
if (choice === 'paste') {
|
if (choice === 'paste') {
|
||||||
io.stdout.write(
|
io.stdout.write(
|
||||||
'│ KTX will save the key in .ktx/secrets/anthropic-api-key with local file permissions, then write a file: reference in ktx.yaml.\n',
|
'│ ktx will save the key in .ktx/secrets/anthropic-api-key with local file permissions, then write a file: reference in ktx.yaml.\n',
|
||||||
);
|
);
|
||||||
const value = await prompts.password({ message: withTextInputNavigation('Anthropic API key') });
|
const value = await prompts.password({ message: withTextInputNavigation('Anthropic API key') });
|
||||||
if (value === undefined) {
|
if (value === undefined) {
|
||||||
|
|
@ -488,7 +488,7 @@ async function chooseBackend(
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
const choice = await prompts.select({
|
const choice = await prompts.select({
|
||||||
message: 'Which LLM provider should KTX use?',
|
message: 'Which LLM provider should ktx use?',
|
||||||
options: [
|
options: [
|
||||||
...KTX_SETUP_LLM_BACKENDS.map((backend) => ({ value: backend, label: KTX_SETUP_LLM_BACKEND_LABELS[backend] })),
|
...KTX_SETUP_LLM_BACKENDS.map((backend) => ({ value: backend, label: KTX_SETUP_LLM_BACKEND_LABELS[backend] })),
|
||||||
{ value: 'back', label: 'Back' },
|
{ value: 'back', label: 'Back' },
|
||||||
|
|
@ -599,7 +599,7 @@ async function chooseInteractiveVertexProject(
|
||||||
}
|
}
|
||||||
|
|
||||||
const choice = await prompts.autocomplete({
|
const choice = await prompts.autocomplete({
|
||||||
message: `Which Google Cloud project should KTX use for Vertex AI?\n\n${[
|
message: `Which Google Cloud project should ktx use for Vertex AI?\n\n${[
|
||||||
VERTEX_PROJECT_PROMPT_CONTEXT,
|
VERTEX_PROJECT_PROMPT_CONTEXT,
|
||||||
listFailureMessage,
|
listFailureMessage,
|
||||||
]
|
]
|
||||||
|
|
|
||||||
|
|
@ -144,7 +144,7 @@ async function confirmProjectDir(
|
||||||
const action = await prompts.select({
|
const action = await prompts.select({
|
||||||
message: `That folder already exists and is not empty: ${selectedDir}`,
|
message: `That folder already exists and is not empty: ${selectedDir}`,
|
||||||
options: [
|
options: [
|
||||||
{ value: 'use-existing', label: 'Yes, create KTX files there' },
|
{ value: 'use-existing', label: 'Yes, create ktx files there' },
|
||||||
{ value: 'choose-another', label: 'Choose another folder' },
|
{ value: 'choose-another', label: 'Choose another folder' },
|
||||||
{ value: 'back', label: 'Back' },
|
{ value: 'back', label: 'Back' },
|
||||||
],
|
],
|
||||||
|
|
@ -155,9 +155,9 @@ async function confirmProjectDir(
|
||||||
return { status: 'confirmed', confirmedCreation: true };
|
return { status: 'confirmed', confirmedCreation: true };
|
||||||
}
|
}
|
||||||
|
|
||||||
io.stdout.write(`│ KTX will create:\n│ ${selectedDir}\n`);
|
io.stdout.write(`│ ktx will create:\n│ ${selectedDir}\n`);
|
||||||
const action = await prompts.select({
|
const action = await prompts.select({
|
||||||
message: `Create KTX project at ${selectedDir}?`,
|
message: `Create ktx project at ${selectedDir}?`,
|
||||||
options: [
|
options: [
|
||||||
{ value: 'create', label: 'Create project' },
|
{ value: 'create', label: 'Create project' },
|
||||||
{ value: 'choose-another', label: 'Choose another folder' },
|
{ value: 'choose-another', label: 'Choose another folder' },
|
||||||
|
|
@ -210,7 +210,7 @@ async function promptForNewProjectDir(
|
||||||
|
|
||||||
while (true) {
|
while (true) {
|
||||||
const destinationChoice = await prompts.select({
|
const destinationChoice = await prompts.select({
|
||||||
message: 'Where should KTX create the project?',
|
message: 'Where should ktx create the project?',
|
||||||
options: [
|
options: [
|
||||||
{ value: 'default', label: `Create the default project folder: ${defaultProjectDir}` },
|
{ value: 'default', label: `Create the default project folder: ${defaultProjectDir}` },
|
||||||
{ value: 'custom', label: 'Enter a custom path' },
|
{ value: 'custom', label: 'Enter a custom path' },
|
||||||
|
|
@ -337,7 +337,7 @@ export async function runKtxSetupProjectStep(
|
||||||
);
|
);
|
||||||
while (true) {
|
while (true) {
|
||||||
const choice = await prompts.select({
|
const choice = await prompts.select({
|
||||||
message: 'Where should KTX create the project?',
|
message: 'Where should ktx create the project?',
|
||||||
options: [
|
options: [
|
||||||
{ value: 'current', label: `Current directory (${projectDir})` },
|
{ value: 'current', label: `Current directory (${projectDir})` },
|
||||||
{ value: 'new-default', label: `New subfolder (${defaultProjectDirLabel})` },
|
{ value: 'new-default', label: `New subfolder (${defaultProjectDirLabel})` },
|
||||||
|
|
|
||||||
|
|
@ -99,7 +99,7 @@ export async function runKtxSetupReadyChangeMenu(
|
||||||
{ value: 'databases', label: 'Databases' },
|
{ value: 'databases', label: 'Databases' },
|
||||||
{ value: 'sources', label: 'Context sources' },
|
{ value: 'sources', label: 'Context sources' },
|
||||||
...(status.runtime.required ? [{ value: 'runtime', label: 'Runtime' }] : []),
|
...(status.runtime.required ? [{ value: 'runtime', label: 'Runtime' }] : []),
|
||||||
{ value: 'context', label: 'Rebuild KTX context' },
|
{ value: 'context', label: 'Rebuild ktx context' },
|
||||||
{ value: 'agents', label: 'Agent integration' },
|
{ value: 'agents', label: 'Agent integration' },
|
||||||
{ value: 'exit', label: 'Exit' },
|
{ value: 'exit', label: 'Exit' },
|
||||||
],
|
],
|
||||||
|
|
|
||||||
|
|
@ -166,7 +166,7 @@ function sourceAdapter(source: KtxSetupSourceType): string {
|
||||||
}
|
}
|
||||||
|
|
||||||
function connectionNamePrompt(label: string): string {
|
function connectionNamePrompt(label: string): string {
|
||||||
return `Name this ${label} connection\nKTX will use this short name in commands and config. You can rename it now.`;
|
return `Name this ${label} connection\nktx will use this short name in commands and config. You can rename it now.`;
|
||||||
}
|
}
|
||||||
|
|
||||||
function sourceSubpathPrompt(source: KtxSetupSourceType): string {
|
function sourceSubpathPrompt(source: KtxSetupSourceType): string {
|
||||||
|
|
@ -266,7 +266,7 @@ async function chooseSourceCredentialRef(input: {
|
||||||
}): Promise<string | 'back'> {
|
}): Promise<string | 'back'> {
|
||||||
while (true) {
|
while (true) {
|
||||||
const choice = await input.prompts.select({
|
const choice = await input.prompts.select({
|
||||||
message: `How should KTX find your ${input.label}?`,
|
message: `How should ktx find your ${input.label}?`,
|
||||||
options: [
|
options: [
|
||||||
...(input.existingRef ? [{ value: 'keep', label: 'Keep existing credential' }] : []),
|
...(input.existingRef ? [{ value: 'keep', label: 'Keep existing credential' }] : []),
|
||||||
{ value: 'paste', label: 'Paste a key and save it as a local secret file' },
|
{ value: 'paste', label: 'Paste a key and save it as a local secret file' },
|
||||||
|
|
@ -1179,7 +1179,7 @@ async function promptForInteractiveSource(
|
||||||
}
|
}
|
||||||
if (subpaths.length > 1) {
|
if (subpaths.length > 1) {
|
||||||
const selected = await prompts.select({
|
const selected = await prompts.select({
|
||||||
message: 'Multiple dbt projects found — which one should KTX use?',
|
message: 'Multiple dbt projects found — which one should ktx use?',
|
||||||
options: [
|
options: [
|
||||||
...subpaths.map((p) => ({ value: p || '.', label: p || '(project root)' })),
|
...subpaths.map((p) => ({ value: p || '.', label: p || '(project root)' })),
|
||||||
{ value: 'back', label: 'Back' },
|
{ value: 'back', label: 'Back' },
|
||||||
|
|
@ -1341,7 +1341,7 @@ async function promptForInteractiveSource(
|
||||||
},
|
},
|
||||||
async (currentState) => {
|
async (currentState) => {
|
||||||
const crawlMode = await prompts.select({
|
const crawlMode = await prompts.select({
|
||||||
message: 'Which Notion pages should KTX ingest?',
|
message: 'Which Notion pages should ktx ingest?',
|
||||||
options: [
|
options: [
|
||||||
{ value: 'all_accessible', label: 'All pages the integration can access' },
|
{ value: 'all_accessible', label: 'All pages the integration can access' },
|
||||||
{ value: 'selected_roots', label: 'Specific pages and their subpages (choose them in a picker)' },
|
{ value: 'selected_roots', label: 'Specific pages and their subpages (choose them in a picker)' },
|
||||||
|
|
@ -1979,7 +1979,7 @@ export async function runKtxSetupSourcesStep(
|
||||||
: args.inputMode === 'disabled'
|
: args.inputMode === 'disabled'
|
||||||
? []
|
? []
|
||||||
: await prompts.multiselect({
|
: await prompts.multiselect({
|
||||||
message: withMultiselectNavigation('Which context sources should KTX ingest?'),
|
message: withMultiselectNavigation('Which context sources should ktx ingest?'),
|
||||||
options: contextSourceChecklist.options,
|
options: contextSourceChecklist.options,
|
||||||
...(contextSourceChecklist.initialValues.length > 0
|
...(contextSourceChecklist.initialValues.length > 0
|
||||||
? { initialValues: contextSourceChecklist.initialValues }
|
? { initialValues: contextSourceChecklist.initialValues }
|
||||||
|
|
|
||||||
|
|
@ -318,16 +318,16 @@ async function runKtxSetupEntryMenu(
|
||||||
const options = status.project.ready
|
const options = status.project.ready
|
||||||
? [
|
? [
|
||||||
{ value: 'setup', label: 'Resume or change an existing setup' },
|
{ value: 'setup', label: 'Resume or change an existing setup' },
|
||||||
{ value: 'new-project', label: 'Create a new KTX project' },
|
{ value: 'new-project', label: 'Create a new ktx project' },
|
||||||
{ value: 'agents', label: 'Connect a coding agent to KTX' },
|
{ value: 'agents', label: 'Connect a coding agent to ktx' },
|
||||||
{ value: 'status', label: 'Check setup status' },
|
{ value: 'status', label: 'Check setup status' },
|
||||||
{ value: 'demo', label: 'Explore a pre-built KTX project' },
|
{ value: 'demo', label: 'Explore a pre-built ktx project' },
|
||||||
{ value: 'exit', label: 'Exit' },
|
{ value: 'exit', label: 'Exit' },
|
||||||
]
|
]
|
||||||
: [
|
: [
|
||||||
{ value: 'setup', label: 'Set up KTX for my data' },
|
{ value: 'setup', label: 'Set up ktx for my data' },
|
||||||
{ value: 'status', label: 'Check setup status' },
|
{ value: 'status', label: 'Check setup status' },
|
||||||
{ value: 'demo', label: 'Explore a pre-built KTX project' },
|
{ value: 'demo', label: 'Explore a pre-built ktx project' },
|
||||||
{ value: 'exit', label: 'Exit' },
|
{ value: 'exit', label: 'Exit' },
|
||||||
];
|
];
|
||||||
const action = (await prompts.select({
|
const action = (await prompts.select({
|
||||||
|
|
@ -523,17 +523,17 @@ function formatContextBuilt(status: KtxSetupContextStatusSummary): string {
|
||||||
export function formatKtxSetupStatus(status: KtxSetupStatus): string {
|
export function formatKtxSetupStatus(status: KtxSetupStatus): string {
|
||||||
if (!status.project.ready) {
|
if (!status.project.ready) {
|
||||||
return [
|
return [
|
||||||
`No KTX project found at ${status.project.path}.`,
|
`No ktx project found at ${status.project.path}.`,
|
||||||
'',
|
'',
|
||||||
'Check another project: ktx --project-dir <folder> status',
|
'Check another project: ktx --project-dir <folder> status',
|
||||||
'Or from that folder: ktx status',
|
'Or from that folder: ktx status',
|
||||||
'Create a new KTX project here: ktx setup',
|
'Create a new ktx project here: ktx setup',
|
||||||
'',
|
'',
|
||||||
].join('\n');
|
].join('\n');
|
||||||
}
|
}
|
||||||
|
|
||||||
const lines = [
|
const lines = [
|
||||||
`KTX project: ${status.project.path}`,
|
`ktx project: ${status.project.path}`,
|
||||||
`Project ready: ${formatReady(status.project.ready)}`,
|
`Project ready: ${formatReady(status.project.ready)}`,
|
||||||
`LLM ready: ${formatReady(status.llm.ready)}${status.llm.model ? ` (${status.llm.model})` : ''}`,
|
`LLM ready: ${formatReady(status.llm.ready)}${status.llm.model ? ` (${status.llm.model})` : ''}`,
|
||||||
`Embeddings ready: ${formatReady(status.embeddings.ready)}${
|
`Embeddings ready: ${formatReady(status.embeddings.ready)}${
|
||||||
|
|
@ -548,7 +548,7 @@ export function formatKtxSetupStatus(status: KtxSetupStatus): string {
|
||||||
}`,
|
}`,
|
||||||
]
|
]
|
||||||
: []),
|
: []),
|
||||||
`KTX context built: ${formatContextBuilt(status.context)}`,
|
`ktx context built: ${formatContextBuilt(status.context)}`,
|
||||||
`Agent integration ready: ${formatReady(status.agents.some((agent) => agent.ready))}${
|
`Agent integration ready: ${formatReady(status.agents.some((agent) => agent.ready))}${
|
||||||
status.agents.length > 0 ? ` (${status.agents.map((agent) => `${agent.target}:${agent.scope}`).join(', ')})` : ''
|
status.agents.length > 0 ? ` (${status.agents.map((agent) => `${agent.target}:${agent.scope}`).join(', ')})` : ''
|
||||||
}`,
|
}`,
|
||||||
|
|
@ -585,7 +585,7 @@ export function formatKtxSetupCompletionSummary(
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
lines.push('', agentNextActions ? 'After that, try' : 'Try it');
|
lines.push('', agentNextActions ? 'After that, try' : 'Try it');
|
||||||
lines.push(' Ask your agent: "Use KTX to show me the available tables."');
|
lines.push(' Ask your agent: "Use ktx to show me the available tables."');
|
||||||
return lines.join('\n');
|
return lines.join('\n');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -622,7 +622,7 @@ function setupRuntimeInstallPolicy(args: Extract<KtxSetupArgs, { command: 'run'
|
||||||
|
|
||||||
async function commitSetupConfigChanges(projectDir: string): Promise<void> {
|
async function commitSetupConfigChanges(projectDir: string): Promise<void> {
|
||||||
const project = await loadKtxProject({ projectDir });
|
const project = await loadKtxProject({ projectDir });
|
||||||
await project.git.commitFile('ktx.yaml', 'setup: update KTX project config', 'ktx setup', 'setup@ktx.local');
|
await project.git.commitFile('ktx.yaml', 'setup: update ktx project config', 'ktx setup', 'setup@ktx.local');
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function runKtxSetup(args: KtxSetupArgs, io: KtxCliIo, deps: KtxSetupDeps = {}): Promise<number> {
|
export async function runKtxSetup(args: KtxSetupArgs, io: KtxCliIo, deps: KtxSetupDeps = {}): Promise<number> {
|
||||||
|
|
@ -638,7 +638,7 @@ export async function runKtxSetup(args: KtxSetupArgs, io: KtxCliIo, deps: KtxSet
|
||||||
|
|
||||||
async function runKtxSetupInner(args: KtxSetupArgs, io: KtxCliIo, deps: KtxSetupDeps = {}): Promise<number> {
|
async function runKtxSetupInner(args: KtxSetupArgs, io: KtxCliIo, deps: KtxSetupDeps = {}): Promise<number> {
|
||||||
const setupUi = deps.setupUi ?? createKtxSetupUiAdapter();
|
const setupUi = deps.setupUi ?? createKtxSetupUiAdapter();
|
||||||
setupUi.intro('KTX setup', io);
|
setupUi.intro('ktx setup', io);
|
||||||
setupUi.note(KTX_DOCS_URL, '📚 Docs', io);
|
setupUi.note(KTX_DOCS_URL, '📚 Docs', io);
|
||||||
let entryAction: KtxSetupEntryAction | undefined;
|
let entryAction: KtxSetupEntryAction | undefined;
|
||||||
let projectResult: Awaited<ReturnType<typeof runKtxSetupProjectStep>>;
|
let projectResult: Awaited<ReturnType<typeof runKtxSetupProjectStep>>;
|
||||||
|
|
@ -989,7 +989,7 @@ async function runKtxSetupInner(args: KtxSetupArgs, io: KtxCliIo, deps: KtxSetup
|
||||||
if (shouldPrintConciseReadySummary(status)) {
|
if (shouldPrintConciseReadySummary(status)) {
|
||||||
setupUi.note(
|
setupUi.note(
|
||||||
formatKtxSetupCompletionSummary(status, { agentNextActions }),
|
formatKtxSetupCompletionSummary(status, { agentNextActions }),
|
||||||
agentNextActions ? 'Finish KTX agent setup' : 'KTX project ready',
|
agentNextActions ? 'Finish ktx agent setup' : 'ktx project ready',
|
||||||
io,
|
io,
|
||||||
{
|
{
|
||||||
format: (line) => line,
|
format: (line) => line,
|
||||||
|
|
|
||||||
|
|
@ -1,11 +1,11 @@
|
||||||
---
|
---
|
||||||
name: ktx-analytics
|
name: ktx-analytics
|
||||||
description: Use when answering a question that needs data from a KTX-connected database - investigating, analyzing, "how many", "show me", "what's the breakdown of", finding records by value, exploring tables, comparing periods, explaining metrics, or any data-analysis request. Triggers even when the user does not say "analytics"; if the answer requires querying a configured KTX connection, this skill applies.
|
description: Use when answering a question that needs data from a ktx-connected database - investigating, analyzing, "how many", "show me", "what's the breakdown of", finding records by value, exploring tables, comparing periods, explaining metrics, or any data-analysis request. Triggers even when the user does not say "analytics"; if the answer requires querying a configured ktx connection, this skill applies.
|
||||||
---
|
---
|
||||||
|
|
||||||
# KTX Analytics Workflow
|
# ktx Analytics Workflow
|
||||||
|
|
||||||
You have access to KTX MCP tools for data discovery, semantic-layer analysis, raw read-only SQL, wiki context, and memory ingest. Follow this workflow.
|
You have access to ktx MCP tools for data discovery, semantic-layer analysis, raw read-only SQL, wiki context, and memory ingest. Follow this workflow.
|
||||||
|
|
||||||
<workflow>
|
<workflow>
|
||||||
1. **Discover** - call `discover_data` first to see what exists across wiki pages, semantic-layer sources, metrics, dimensions, raw tables, and columns. Returns refs only.
|
1. **Discover** - call `discover_data` first to see what exists across wiki pages, semantic-layer sources, metrics, dimensions, raw tables, and columns. Returns refs only.
|
||||||
|
|
|
||||||
|
|
@ -1,16 +1,16 @@
|
||||||
---
|
---
|
||||||
name: dbt_ingest
|
name: dbt_ingest
|
||||||
description: Map dbt `schema.yml` / `properties.yml` models and sources into KTX semantic-layer overlays and column notes. Covers `sources:` vs `models:`, column `data_tests` (not_null, unique, accepted_values, relationships), and how bundle-time writes complement manifest backfill from git sync. Load when the WorkUnit's `skillNames` includes `dbt_ingest` or when raw files are dbt YAML under `models/` / `sources/`.
|
description: Map dbt `schema.yml` / `properties.yml` models and sources into ktx semantic-layer overlays and column notes. Covers `sources:` vs `models:`, column `data_tests` (not_null, unique, accepted_values, relationships), and how bundle-time writes complement manifest backfill from git sync. Load when the WorkUnit's `skillNames` includes `dbt_ingest` or when raw files are dbt YAML under `models/` / `sources/`.
|
||||||
callers: [memory_agent]
|
callers: [memory_agent]
|
||||||
---
|
---
|
||||||
|
|
||||||
# dbt → KTX (bundle ingest)
|
# dbt → ktx (bundle ingest)
|
||||||
|
|
||||||
Use this skill for **uploaded** dbt projects (`dbt_project.yml` at stage root, `models/**`, `sources/**`, `schema.yml`). There is **no** `fetch()` in v1 - scheduled `dbt parse` / `manifest.json` pulls are out of scope; host-provided dbt sync may still backfill structured test metadata into `_schema` on the next sync.
|
Use this skill for **uploaded** dbt projects (`dbt_project.yml` at stage root, `models/**`, `sources/**`, `schema.yml`). There is **no** `fetch()` in v1 - scheduled `dbt parse` / `manifest.json` pulls are out of scope; host-provided dbt sync may still backfill structured test metadata into `_schema` on the next sync.
|
||||||
|
|
||||||
## Mapping (models / sources → SL)
|
## Mapping (models / sources → SL)
|
||||||
|
|
||||||
| dbt | KTX | Notes |
|
| dbt | ktx | Notes |
|
||||||
|-----|--------|--------|
|
|-----|--------|--------|
|
||||||
| `models:` entry with `columns:` | **Overlay** on the manifest table with the same name (after `discover_data` / `entity_details`) | One SL source per physical table; model name may differ from DB name - resolve with `read_raw_file` + warehouse context. |
|
| `models:` entry with `columns:` | **Overlay** on the manifest table with the same name (after `discover_data` / `entity_details`) | One SL source per physical table; model name may differ from DB name - resolve with `read_raw_file` + warehouse context. |
|
||||||
| `sources:` → `tables:` | Same as models; use `identifier` when present instead of logical `name`. | Schema + name must match how the connection sees tables. |
|
| `sources:` → `tables:` | Same as models; use `identifier` when present instead of logical `name`. | Schema + name must match how the connection sees tables. |
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,12 @@
|
||||||
---
|
---
|
||||||
name: looker_ingest
|
name: looker_ingest
|
||||||
description: Extract durable KTX knowledge and semantic-layer contribution proposals from staged Looker runtime dashboard, Look, and explore JSON. Load for WorkUnits whose raw files are under explores/, dashboards/, or looks/.
|
description: Extract durable ktx knowledge and semantic-layer contribution proposals from staged Looker runtime dashboard, Look, and explore JSON. Load for WorkUnits whose raw files are under explores/, dashboards/, or looks/.
|
||||||
callers: [memory_agent]
|
callers: [memory_agent]
|
||||||
---
|
---
|
||||||
|
|
||||||
# Looker Runtime Ingest
|
# Looker Runtime Ingest
|
||||||
|
|
||||||
Looker runtime ingest turns API-staged dashboards, Looks, and explores into durable KTX memory. Runtime entities are evidence. They are not themselves the final knowledge shape.
|
Looker runtime ingest turns API-staged dashboards, Looks, and explores into durable ktx memory. Runtime entities are evidence. They are not themselves the final knowledge shape.
|
||||||
|
|
||||||
## Required Workflow
|
## Required Workflow
|
||||||
|
|
||||||
|
|
@ -103,7 +103,7 @@ The staged explore file carries warehouse target fields populated before the WU
|
||||||
- `rawSqlTableName`: Looker's verbatim `sql_table_name`. Keep it as provenance only.
|
- `rawSqlTableName`: Looker's verbatim `sql_table_name`. Keep it as provenance only.
|
||||||
- `targetTable`: the parsed target-table union. Use this as the sole branch condition.
|
- `targetTable`: the parsed target-table union. Use this as the sole branch condition.
|
||||||
|
|
||||||
When `targetTable.ok === true`, the explore has a complete KTX backing target. Before writing:
|
When `targetTable.ok === true`, the explore has a complete ktx backing target. Before writing:
|
||||||
|
|
||||||
1. Use `targetTable.catalog`, `targetTable.schema`, and `targetTable.name` for `source_tables` preflight matching through `sl_discover` or `sl_read_source`.
|
1. Use `targetTable.catalog`, `targetTable.schema`, and `targetTable.name` for `source_tables` preflight matching through `sl_discover` or `sl_read_source`.
|
||||||
2. Use Looker field `sql`, labels, descriptions, and type metadata to derive source columns, measures, segments, joins, and grain.
|
2. Use Looker field `sql`, labels, descriptions, and type metadata to derive source columns, measures, segments, joins, and grain.
|
||||||
|
|
|
||||||
|
|
@ -1,16 +1,16 @@
|
||||||
---
|
---
|
||||||
name: lookml_ingest
|
name: lookml_ingest
|
||||||
description: Map a LookML view/model/explore into KTX semantic layer sources. Covers the LookML to KTX primitive table, provenance tagging, and three worked examples (overlay, standalone from derived_table, standalone with sql_always_where). Load when the turn contains `.lkml` content.
|
description: Map a LookML view/model/explore into ktx semantic layer sources. Covers the LookML to ktx primitive table, provenance tagging, and three worked examples (overlay, standalone from derived_table, standalone with sql_always_where). Load when the turn contains `.lkml` content.
|
||||||
callers: [memory_agent]
|
callers: [memory_agent]
|
||||||
---
|
---
|
||||||
|
|
||||||
# LookML to KTX Semantic Layer
|
# LookML to ktx Semantic Layer
|
||||||
|
|
||||||
LookML views map to SL sources, `measure:` to measures, `explore: { join: }` to the join graph. This skill lays out the mapping and the three capture shapes.
|
LookML views map to SL sources, `measure:` to measures, `explore: { join: }` to the join graph. This skill lays out the mapping and the three capture shapes.
|
||||||
|
|
||||||
## Mapping table
|
## Mapping table
|
||||||
|
|
||||||
| LookML | KTX form | Notes |
|
| LookML | ktx form | Notes |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `view: X { sql_table_name: …; measure:/dimension:/join: }` | **Overlay** named `X` with `measures`, computed-only `columns`, `column_overrides`, `joins`, `segments` | Manifest-backed; inherit grain/columns |
|
| `view: X { sql_table_name: …; measure:/dimension:/join: }` | **Overlay** named `X` with `measures`, computed-only `columns`, `column_overrides`, `joins`, `segments` | Manifest-backed; inherit grain/columns |
|
||||||
| `view: X { derived_table: { sql: … } }` | **Standalone** with top-level `sql:`, explicit `grain:` + `columns:` | No manifest entry exists |
|
| `view: X { derived_table: { sql: … } }` | **Standalone** with top-level `sql:`, explicit `grain:` + `columns:` | No manifest entry exists |
|
||||||
|
|
@ -23,7 +23,7 @@ Type map: `date`/`datetime`/`timestamp` → `time`; `yesno` → `boolean`; `numb
|
||||||
|
|
||||||
## Decision rules
|
## Decision rules
|
||||||
|
|
||||||
LookML writes target the run connection directly. Unlike Looker runtime ingestion, the LookML adapter is configured on the warehouse KTX connection, so do not look for `targetWarehouseConnectionId` and do not route through a mapping array.
|
LookML writes target the run connection directly. Unlike Looker runtime ingestion, the LookML adapter is configured on the warehouse ktx connection, so do not look for `targetWarehouseConnectionId` and do not route through a mapping array.
|
||||||
|
|
||||||
Before any SL write, inspect the WorkUnit notes.
|
Before any SL write, inspect the WorkUnit notes.
|
||||||
|
|
||||||
|
|
@ -94,7 +94,7 @@ SL source, `tables:` frontmatter, `sl_refs`, or `emit_unmapped_fallback`:
|
||||||
schema or dataset, and table from the WorkUnit evidence.
|
schema or dataset, and table from the WorkUnit evidence.
|
||||||
3. Use only those names in `sql:`, `columns:`, and `grain:`. Map each `dimension_group` to ONE `{ name: <physical_col>, type: time, role: time }` entry - never one per timeframe.
|
3. Use only those names in `sql:`, `columns:`, and `grain:`. Map each `dimension_group` to ONE `{ name: <physical_col>, type: time, role: time }` entry - never one per timeframe.
|
||||||
|
|
||||||
| LookML input | KTX `columns:` entry |
|
| LookML input | ktx `columns:` entry |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `dimension_group: month { type: time; timeframes: [month]; sql: ${TABLE}.month_date ;; }` | `{ name: month_date, type: time, role: time }` |
|
| `dimension_group: month { type: time; timeframes: [month]; sql: ${TABLE}.month_date ;; }` | `{ name: month_date, type: time, role: time }` |
|
||||||
| `dimension_group: date { type: time; timeframes: [raw, date, week, month]; sql: ${TABLE}.date ;; }` | `{ name: date, type: time, role: time }` - single entry, NOT `date_raw`/`date_date`/`date_week` |
|
| `dimension_group: date { type: time; timeframes: [raw, date, week, month]; sql: ${TABLE}.date ;; }` | `{ name: date, type: time, role: time }` - single entry, NOT `date_raw`/`date_date`/`date_week` |
|
||||||
|
|
@ -132,7 +132,7 @@ explore: fct_labs {
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
KTX overlay at `<connId>/fct_labs.yaml`:
|
ktx overlay at `<connId>/fct_labs.yaml`:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
name: fct_labs
|
name: fct_labs
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,12 @@
|
||||||
---
|
---
|
||||||
name: metabase_ingest
|
name: metabase_ingest
|
||||||
description: Convert Metabase questions, models, and metrics into KTX Semantic Layer source definitions. Covers result-metadata to KSL column type mapping, FK/PK detection, near-duplicate deduplication, pre-aggregation decomposition, join-graph connectivity, and how to react to priorProvenance from earlier ingest syncs. Load when the WorkUnit contains `cards/<id>.json` files under a Metabase bundle.
|
description: Convert Metabase questions, models, and metrics into ktx Semantic Layer source definitions. Covers result-metadata to KSL column type mapping, FK/PK detection, near-duplicate deduplication, pre-aggregation decomposition, join-graph connectivity, and how to react to priorProvenance from earlier ingest syncs. Load when the WorkUnit contains `cards/<id>.json` files under a Metabase bundle.
|
||||||
callers: [memory_agent]
|
callers: [memory_agent]
|
||||||
---
|
---
|
||||||
|
|
||||||
# Metabase to KTX Semantic Layer
|
# Metabase to ktx Semantic Layer
|
||||||
|
|
||||||
Each WorkUnit represents one Metabase collection's cards for one Metabase database (mapped to exactly one KTX connection). Every `cards/<id>.json` file carries the resolved SQL, result_metadata, card type, collection path, and referenced-card ids. The WU's `sync-config.json` tells you which sync mode is active and which selections apply. `databases/<id>.json` tells you the target KTX connection.
|
Each WorkUnit represents one Metabase collection's cards for one Metabase database (mapped to exactly one ktx connection). Every `cards/<id>.json` file carries the resolved SQL, result_metadata, card type, collection path, and referenced-card ids. The WU's `sync-config.json` tells you which sync mode is active and which selections apply. `databases/<id>.json` tells you the target ktx connection.
|
||||||
|
|
||||||
## Context format
|
## Context format
|
||||||
|
|
||||||
|
|
@ -100,7 +100,7 @@ measures:
|
||||||
|
|
||||||
Overlay shape: `name:` plus any of `measures:`, `segments:`, `descriptions:`, `joins:`, `disable_joins:`, `exclude_columns:`, `column_overrides:`, or computed-only `columns:` entries with `expr` + `type`. Never include `sql:`, `table:`, `grain:`, or base-table `columns:` on a manifest-backed name — those would shadow the manifest's schema and drop its joins. Use `column_overrides:` for inherited column descriptions. Overlay `joins:` are merged additively with the manifest's joins (deduped by `to` + `on`); use `disable_joins: ["<on-clause>"]` to suppress a specific manifest join. After the overlay exists, use `sl_edit_source` for further tweaks. See `sl_capture` skill for the canonical overlay rule.
|
Overlay shape: `name:` plus any of `measures:`, `segments:`, `descriptions:`, `joins:`, `disable_joins:`, `exclude_columns:`, `column_overrides:`, or computed-only `columns:` entries with `expr` + `type`. Never include `sql:`, `table:`, `grain:`, or base-table `columns:` on a manifest-backed name — those would shadow the manifest's schema and drop its joins. Use `column_overrides:` for inherited column descriptions. Overlay `joins:` are merged additively with the manifest's joins (deduped by `to` + `on`); use `disable_joins: ["<on-clause>"]` to suppress a specific manifest join. After the overlay exists, use `sl_edit_source` for further tweaks. See `sl_capture` skill for the canonical overlay rule.
|
||||||
|
|
||||||
**Join discovery:** When your card's SQL references warehouse tables (e.g. in `FROM` or `JOIN` clauses), call `sl_discover({ query: '<table>' })` before writing. The matching manifest entry's `name` is the value you use in `joins: [- to: <name>]` only when the card output exposes a local key that matches the target source grain (for example `account_id = mart_account_segments.account_id`). Do not declare a KTX join just because the card SQL joins that table internally. If the output only exposes display fields such as `account_name`, keep the SQL source self-contained or project the key before adding the join. Use `many_to_one` for FK-to-dimension joins, `one_to_many` for the reverse.
|
**Join discovery:** When your card's SQL references warehouse tables (e.g. in `FROM` or `JOIN` clauses), call `sl_discover({ query: '<table>' })` before writing. The matching manifest entry's `name` is the value you use in `joins: [- to: <name>]` only when the card output exposes a local key that matches the target source grain (for example `account_id = mart_account_segments.account_id`). Do not declare a ktx join just because the card SQL joins that table internally. If the output only exposes display fields such as `account_name`, keep the SQL source self-contained or project the key before adding the join. Use `many_to_one` for FK-to-dimension joins, `one_to_many` for the reverse.
|
||||||
|
|
||||||
**Hard rule on join columns (prevents broken joins):** For every join you declare, the local column on the left of `on:` MUST be (a) present in your source's projected output and (b) a key/ID column, never a display value. If the natural FK isn't in your SELECT, add it to SELECT before declaring the join. Joining `account_name = mart_account_segments.account_id` is always wrong - names are not identifiers and the equality produces zero matches. The validator rejects this with a "display value to identifier" error; the tool will refuse to save it. Add `account_id` to your SELECT and join on `account_id = mart_account_segments.account_id`, or omit the join entirely.
|
**Hard rule on join columns (prevents broken joins):** For every join you declare, the local column on the left of `on:` MUST be (a) present in your source's projected output and (b) a key/ID column, never a display value. If the natural FK isn't in your SELECT, add it to SELECT before declaring the join. Joining `account_name = mart_account_segments.account_id` is always wrong - names are not identifiers and the equality produces zero matches. The validator rejects this with a "display value to identifier" error; the tool will refuse to save it. Add `account_id` to your SELECT and join on `account_id = mart_account_segments.account_id`, or omit the join entirely.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,16 +1,16 @@
|
||||||
---
|
---
|
||||||
name: metricflow_ingest
|
name: metricflow_ingest
|
||||||
description: Map a MetricFlow semantic_model or metric into KTX semantic layer sources. Covers the MetricFlow to KTX primitive table, `extends:` inheritance flattening, metric-type handling (simple / derived / ratio / cumulative / conversion), `model: ref('x')` resolution, and four worked examples. Load when the turn contains `.yml`/`.yaml` files with top-level `semantic_models:` or `metrics:`.
|
description: Map a MetricFlow semantic_model or metric into ktx semantic layer sources. Covers the MetricFlow to ktx primitive table, `extends:` inheritance flattening, metric-type handling (simple / derived / ratio / cumulative / conversion), `model: ref('x')` resolution, and four worked examples. Load when the turn contains `.yml`/`.yaml` files with top-level `semantic_models:` or `metrics:`.
|
||||||
callers: [memory_agent]
|
callers: [memory_agent]
|
||||||
---
|
---
|
||||||
|
|
||||||
# MetricFlow to KTX Semantic Layer
|
# MetricFlow to ktx Semantic Layer
|
||||||
|
|
||||||
A MetricFlow `semantic_model` maps to an SL source; MetricFlow `measures` map to KTX measures; MetricFlow `entities` map to KTX `joins`; MetricFlow `metrics` (top-level) map to KTX measures OR to cross-model derived measures. Files in one WorkUnit are ALWAYS part of the same logical entity (a connected component, possibly spanning `extends:` + cross-model metric refs). Flatten inheritance and cross-file references at write time.
|
A MetricFlow `semantic_model` maps to an SL source; MetricFlow `measures` map to ktx measures; MetricFlow `entities` map to ktx `joins`; MetricFlow `metrics` (top-level) map to ktx measures OR to cross-model derived measures. Files in one WorkUnit are ALWAYS part of the same logical entity (a connected component, possibly spanning `extends:` + cross-model metric refs). Flatten inheritance and cross-file references at write time.
|
||||||
|
|
||||||
## Mapping table
|
## Mapping table
|
||||||
|
|
||||||
| MetricFlow | KTX form | Notes |
|
| MetricFlow | ktx form | Notes |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `semantic_model: X { model: ref('t') }` with measures + dimensions | **Overlay** named `X` with `measures`, computed-only `columns`, `column_overrides`, `joins` | The `model:` ref resolves to a manifest table. |
|
| `semantic_model: X { model: ref('t') }` with measures + dimensions | **Overlay** named `X` with `measures`, computed-only `columns`, `column_overrides`, `joins` | The `model:` ref resolves to a manifest table. |
|
||||||
| `semantic_model: X { model: source('s','t') }` | **Overlay** named `X` over table `t`. | Same shape; `source()` still resolves to a physical table. |
|
| `semantic_model: X { model: source('s','t') }` | **Overlay** named `X` over table `t`. | Same shape; `source()` still resolves to a physical table. |
|
||||||
|
|
@ -23,11 +23,11 @@ A MetricFlow `semantic_model` maps to an SL source; MetricFlow `measures` map to
|
||||||
| `metrics: [{ type: simple, filter: <jinja> }]` | **New measure** on the same source, with the filter translated to SQL and attached via `filter:` | Translate Jinja `{{ Dimension('x__y') }}` to the column name `y`. |
|
| `metrics: [{ type: simple, filter: <jinja> }]` | **New measure** on the same source, with the filter translated to SQL and attached via `filter:` | Translate Jinja `{{ Dimension('x__y') }}` to the column name `y`. |
|
||||||
| `metrics: [{ type: derived, type_params: { expr, metrics } }]` | **Derived measure** on whichever source owns the referenced measures, with `expr:` referencing measure names | If the metric spans models, still write it once on the source owning the "primary" measure (the one the agent judges most central). Mention the cross-model chain in the description. |
|
| `metrics: [{ type: derived, type_params: { expr, metrics } }]` | **Derived measure** on whichever source owns the referenced measures, with `expr:` referencing measure names | If the metric spans models, still write it once on the source owning the "primary" measure (the one the agent judges most central). Mention the cross-model chain in the description. |
|
||||||
| `metrics: [{ type: ratio, type_params: { numerator, denominator } }]` | Same as derived; `expr: "numerator / NULLIF(denominator, 0)"` if no explicit expr | Safe-division by default. |
|
| `metrics: [{ type: ratio, type_params: { numerator, denominator } }]` | Same as derived; `expr: "numerator / NULLIF(denominator, 0)"` if no explicit expr | Safe-division by default. |
|
||||||
| `metrics: [{ type: cumulative, type_params: { window, grain_to_date } }]` | **Standalone** source with a window-function SQL; reference the resulting column as a normal measure | KTX SL has no first-class cumulative primitive (spec Non-goals). |
|
| `metrics: [{ type: cumulative, type_params: { window, grain_to_date } }]` | **Standalone** source with a window-function SQL; reference the resulting column as a normal measure | ktx SL has no first-class cumulative primitive (spec Non-goals). |
|
||||||
| `metrics: [{ type: conversion }]` | **Flag for human** - do NOT write. Emit a wiki note describing the intended semantics. | No KTX equivalent in v1. |
|
| `metrics: [{ type: conversion }]` | **Flag for human** - do NOT write. Emit a wiki note describing the intended semantics. | No ktx equivalent in v1. |
|
||||||
| Metric not mappable | Wiki page `<metric_name>-definition.md` with the full YAML body quoted | Capture the intent even if we can't emit SL. |
|
| Metric not mappable | Wiki page `<metric_name>-definition.md` with the full YAML body quoted | Capture the intent even if we can't emit SL. |
|
||||||
|
|
||||||
Type map: MetricFlow `time` to KTX `time`; `categorical` to `string`; `number` to `number`; `boolean` to `boolean`. Follow `expr` over `name` when both differ - `expr` is the physical column.
|
Type map: MetricFlow `time` to ktx `time`; `categorical` to `string`; `number` to `number`; `boolean` to `boolean`. Follow `expr` over `name` when both differ - `expr` is the physical column.
|
||||||
|
|
||||||
Verify each MetricFlow model source table with entity_details before producing
|
Verify each MetricFlow model source table with entity_details before producing
|
||||||
the corresponding sl_write_source.
|
the corresponding sl_write_source.
|
||||||
|
|
@ -92,7 +92,7 @@ After every `sl_write_source`, call `sl_validate`. The warehouse will reject inv
|
||||||
|
|
||||||
## Cumulative metrics - sql-standalone fallback
|
## Cumulative metrics - sql-standalone fallback
|
||||||
|
|
||||||
KTX SL has no first-class `window:` or `grain_to_date:` primitive in v1 (spec Non-goals). Translate a MetricFlow cumulative metric to a standalone SL source with a window-function SQL:
|
ktx SL has no first-class `window:` or `grain_to_date:` primitive in v1 (spec Non-goals). Translate a MetricFlow cumulative metric to a standalone SL source with a window-function SQL:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# MetricFlow input:
|
# MetricFlow input:
|
||||||
|
|
@ -105,7 +105,7 @@ metrics:
|
||||||
```
|
```
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# KTX standalone output:
|
# ktx standalone output:
|
||||||
name: cum_revenue_7d
|
name: cum_revenue_7d
|
||||||
source_type: sql
|
source_type: sql
|
||||||
sql: |
|
sql: |
|
||||||
|
|
@ -143,7 +143,7 @@ Do NOT emit SL for this. Instead:
|
||||||
- Write a wiki page at `wiki/global/<metric_name>-intent.md` quoting the full YAML body and a one-line explanation of the intended semantics (base event → conversion event within window).
|
- Write a wiki page at `wiki/global/<metric_name>-intent.md` quoting the full YAML body and a one-line explanation of the intended semantics (base event → conversion event within window).
|
||||||
- Call `emit_unmapped_fallback` with `rawPath` set to the MetricFlow file path, `reason: "conversion_metric_unsupported"`, and `fallback: "flagged"`.
|
- Call `emit_unmapped_fallback` with `rawPath` set to the MetricFlow file path, `reason: "conversion_metric_unsupported"`, and `fallback: "flagged"`.
|
||||||
|
|
||||||
When KTX SL gains conversion primitives, re-ingesting will find the prior wiki note (via `priorProvenance`) and replace it with an SL source.
|
When ktx SL gains conversion primitives, re-ingesting will find the prior wiki note (via `priorProvenance`) and replace it with an SL source.
|
||||||
|
|
||||||
## Provenance markers
|
## Provenance markers
|
||||||
|
|
||||||
|
|
@ -174,7 +174,7 @@ semantic_models:
|
||||||
```
|
```
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# KTX overlay at <connId>/orders.yaml:
|
# ktx overlay at <connId>/orders.yaml:
|
||||||
# <!-- from: raw-sources/.../models/orders.yml#L1-10 -->
|
# <!-- from: raw-sources/.../models/orders.yml#L1-10 -->
|
||||||
name: orders
|
name: orders
|
||||||
descriptions:
|
descriptions:
|
||||||
|
|
@ -217,7 +217,7 @@ metrics:
|
||||||
```
|
```
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# KTX overlay at <connId>/orders_ext.yaml (one file; inheritance flattened):
|
# ktx overlay at <connId>/orders_ext.yaml (one file; inheritance flattened):
|
||||||
# <!-- from: raw-sources/.../models/orders.yml#L1-10 -->
|
# <!-- from: raw-sources/.../models/orders.yml#L1-10 -->
|
||||||
# <!-- from: raw-sources/.../models/orders_ext.yml#L1-8 -->
|
# <!-- from: raw-sources/.../models/orders_ext.yml#L1-8 -->
|
||||||
# <!-- from: raw-sources/.../metrics/orders_final.yml#L1-10 -->
|
# <!-- from: raw-sources/.../metrics/orders_final.yml#L1-10 -->
|
||||||
|
|
@ -256,7 +256,7 @@ metrics:
|
||||||
metrics: [{name: revenue}, {name: cost}]
|
metrics: [{name: revenue}, {name: cost}]
|
||||||
```
|
```
|
||||||
|
|
||||||
Because the WorkUnit bundles all three files (cross-component union via the metric), write the derived measure on ONE of the two sources - pick the source whose domain "owns" the metric (here, `sales` - margin is inherently a sales metric). Cross-source references aren't native in KTX SL; treat the metric's operands as already-resolvable in the target source's query context OR emit a standalone SQL that joins the two tables:
|
Because the WorkUnit bundles all three files (cross-component union via the metric), write the derived measure on ONE of the two sources - pick the source whose domain "owns" the metric (here, `sales` - margin is inherently a sales metric). Cross-source references aren't native in ktx SL; treat the metric's operands as already-resolvable in the target source's query context OR emit a standalone SQL that joins the two tables:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# <connId>/sales.yaml
|
# <connId>/sales.yaml
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue