ktx/packages/cli/test/connectors/duckdb/federated-attach.test.ts
Kevin Messiaen 3c4fcc27c7
feat: Add duckdb connector (#308)
* refactor(duckdb): extract shared json-safe bigint helper

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(duckdb): add and register the duckdb primary connector

Add KtxDuckDbDialect, KtxDuckDbScanConnector (local file-backed, read-only,
never-create, main-schema introspection via information_schema and
duckdb_constraints() for foreign keys), and register the duckdb driver across
the dialect factory, driver registry, connection-type enum, warehouse descriptor,
config schema, scan normalization, connection test drivers, and status display.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(duckdb): route live-database ingest through the DuckDB connector

Add the DuckDB live-database introspection bridge and dispatch duckdb
connections to it in local-adapters, matching the SQLite path. Repoint the
config-rejection test off duckdb (now a valid driver) onto the no-driver case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(duckdb): add duckdb to the setup database flow

Offer DuckDB in the interactive checklist and via ktx setup --database duckdb,
with a file-path prompt and duckdb-local default connection id, parallel to SQLite.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(duckdb): attach native duckdb files in federation

Native .duckdb members ATTACH with (READ_ONLY) and no TYPE/INSTALL/LOAD, since
the duckdb format needs no extension. attachTypeForDriver returns null for the
native case; buildAttachStatements builds load statements from non-null types
only and emits a conditional ATTACH clause.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(duckdb): document the duckdb primary-source connector

Add a DuckDB section to the primary-sources integration page (config, read-only
never-create behavior, main-schema scope, federation) and update the
supported-driver assertion in dialects.test.ts to include duckdb.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(duckdb): use single-namespace display shape for main-only refs

DuckDB v1 introspects the main schema and sets db=null on every table, so its
display refs are single-namespace like SQLite. The ansi shape emitted a 1-part
table display it then refused to parse, breaking column-level display resolution.
Switch the dialect to the sqlite display shape and add a round-trip test plus a
composite-foreign-key test that were missing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(duckdb): resolve connector dialect via getDialectForDriver

Route the connector's dialect through the shared factory like every other
connector, now that duckdb is registered. Single construction path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(duckdb): skip schema picker for single-file duckdb setup

DuckDB is a single-file, single-namespace ('main') database like SQLite,
but the setup scope step only skipped the schema picker for sqlite. DuckDB
fell into the multi-schema path with an empty schema list, rendering a
broken picker ("No matches found" for main). Extend the file-based-driver
early-return to cover duckdb so it ingests every table directly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(duckdb): reuse shared config helper and derive scope skip

Route duckdb path resolution through the shared resolveStringReference
helper instead of a local third copy of env:/file: handling. Derive the
setup scope-picker skip from SCOPE_DISCOVERY_SPECS membership rather than
a hardcoded sqlite/duckdb driver list.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(duckdb): use a genuinely unknown driver in the rejection test

The merged "rejects unknown drivers" test used `driver: duckdb` as its
unknown-driver stand-in, which stopped being unknown once this branch
added the duckdb connector. Switch to `nonsense` so it again exercises
the unsupported-driver config error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(duckdb): cover dialect, connector, and live-introspection branches

Codecov flagged uncovered branches as dead code; all are real connector,
dialect, and live-ingest behavior. Add unit tests instead of removing them.

- dialect: precedence ladder, sample/clause builders, profiling expressions
- connector: url/env config forms, error throws, never-create guard,
  cardinality cap branches, table-scope empty/non-empty paths
- live-introspection: full-schema and table-scope extraction

Functions 100%, lines ~99% across the duckdb connector dir.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs: add DuckDB to supported-driver references

The DuckDB connector PR documented the connector itself but left the
scattered supported-driver enumerations stale. Add duckdb to the
federation concept page (participation table, activation, table naming,
limitations), the ktx setup CLI reference, the ktx.yaml warehouse-driver
table, the primary-sources field reference, and the quickstart driver
list (which also restores the missing ClickHouse entry).

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andrey Avtomonov <andreybavt@gmail.com>
2026-07-01 12:06:02 +00:00

151 lines
5.3 KiB
TypeScript

import { mkdtempSync, writeFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { describe, expect, it } from 'vitest';
import { federatedAttachTarget } from '../../../src/connectors/duckdb/federated-attach.js';
import type { FederatedMember } from '../../../src/context/connections/federation.js';
const member = (over: Partial<FederatedMember>): FederatedMember => ({
connectionId: 'm',
driver: 'sqlite',
projectDir: '/proj',
connection: { driver: 'sqlite' },
...over,
});
describe('federatedAttachTarget', () => {
it('resolves a sqlite path: config to an absolute filesystem path against projectDir', () => {
const dir = mkdtempSync(join(tmpdir(), 'ktx-attach-'));
writeFileSync(join(dir, 'reviews.db'), '');
try {
const target = federatedAttachTarget(
member({ driver: 'sqlite', projectDir: dir, connection: { driver: 'sqlite', path: './reviews.db' } }),
{},
);
expect(target).toBe(join(dir, 'reviews.db'));
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
it('resolves a sqlite file:// url to a filesystem path', () => {
const target = federatedAttachTarget(
member({ driver: 'sqlite', connection: { driver: 'sqlite', url: 'file:///data/reviews.db' } }),
{},
);
expect(target).toBe('/data/reviews.db');
});
it('builds a libpq connection string for postgres from host/database/user', () => {
const target = federatedAttachTarget(
member({
driver: 'postgres',
connection: { driver: 'postgres', host: 'h', port: 5433, database: 'books', username: 'u', password: 'p' },
}),
{},
);
expect(target).toContain('host=h');
expect(target).toContain('port=5433');
expect(target).toContain('dbname=books');
expect(target).toContain('user=u');
expect(target).toContain('password=p');
});
it('passes a postgres url through as the connection string', () => {
const target = federatedAttachTarget(
member({ driver: 'postgres', connection: { driver: 'postgres', url: 'env:PG_URL' } }),
{ PG_URL: 'postgresql://localhost/books' },
);
expect(target).toBe('postgresql://localhost/books');
});
it('adds sslmode=require to a postgres url when the member sets ssl', () => {
const target = federatedAttachTarget(
member({ driver: 'postgres', connection: { driver: 'postgres', url: 'env:PG_URL', ssl: true } }),
{ PG_URL: 'postgresql://localhost/books' },
);
expect(target).toContain('sslmode=require');
});
it('keeps a stronger sslmode already pinned in a postgres url', () => {
const target = federatedAttachTarget(
member({ driver: 'postgres', connection: { driver: 'postgres', url: 'env:PG_URL', ssl: true } }),
{ PG_URL: 'postgresql://localhost/books?sslmode=verify-full' },
);
expect(target).toContain('sslmode=verify-full');
expect(target).not.toContain('sslmode=require');
});
it('builds a mysql connection string from host/database/user', () => {
const target = federatedAttachTarget(
member({
driver: 'mysql',
connection: { driver: 'mysql', host: 'h', port: 3307, database: 'app', username: 'u', password: 'p' },
}),
{},
);
expect(target).toContain('host=h');
expect(target).toContain('port=3307');
expect(target).toContain('database=app');
expect(target).toContain('user=u');
expect(target).toContain('password=p');
});
it('quotes mysql values containing spaces', () => {
const target = federatedAttachTarget(
member({
driver: 'mysql',
connection: { driver: 'mysql', host: 'h', database: 'app', username: 'u', password: 'pass word' }, // pragma: allowlist secret
}),
{},
);
expect(target).toContain("password='pass word'"); // pragma: allowlist secret
});
it('emits sslmode=require for a postgres member configured with discrete fields and ssl', () => {
const target = federatedAttachTarget(
member({
driver: 'postgres',
connection: { driver: 'postgres', host: 'h', database: 'db', username: 'u', ssl: true },
}),
{},
);
expect(target).toContain('sslmode=require');
});
it('passes through the postgres search_path as options', () => {
const target = federatedAttachTarget(
member({
driver: 'postgres',
connection: { driver: 'postgres', host: 'h', database: 'db', username: 'u', schema: 'analytics' },
}),
{},
);
expect(target).toContain('search_path=analytics');
});
it('emits ssl_mode=REQUIRED for a mysql member with ssl', () => {
const target = federatedAttachTarget(
member({
driver: 'mysql',
connection: { driver: 'mysql', host: 'h', database: 'db', username: 'u', ssl: true },
}),
{},
);
expect(target).toContain('ssl_mode=REQUIRED');
});
it('resolves a duckdb member to its database file path', () => {
const target = federatedAttachTarget(
{ connectionId: 'dux', driver: 'duckdb', projectDir: '/p', connection: { driver: 'duckdb', path: 'a.duckdb' } },
{},
);
expect(target).toBe('/p/a.duckdb');
});
it('throws for an unsupported driver', () => {
expect(() => federatedAttachTarget(member({ driver: 'snowflake', connection: { driver: 'snowflake' } }), {})).toThrow(
/cannot be attached/i,
);
});
});