mirror of
https://github.com/Kaelio/ktx.git
synced 2026-07-01 08:59:39 +02:00
* refactor(connectors): split KtxDialect into core and KtxSqlDialect Separate the dialect contract into a driver-agnostic core (display/ref formatting and type mapping) and a SQL-only extension (query generators). The catalog and entity-details paths resolve the core dialect for any snapshot driver, so it must stay free of SQL generation; this is the prerequisite refactor for adding non-SQL primary sources. - KtxDialect keeps type, formatDisplayRef, parseDisplayRef, columnDisplayTablePartCount, mapDataType, mapToDimensionType - KtxSqlDialect extends it with quoteIdentifier, formatTableName, and the query/sample/statistics generators; the 7 SQL dialects implement it - add getSqlDialectForDriver for SQL drivers; the 7 connectors and the relationship-benchmark harness consume it - thread the relationship pipeline (profiling/validation/composite/ discovery) as KtxSqlDialect | null so a non-SQL source skips coverage SQL and its candidates stay in review; local-enrichment builds the SQL dialect only when the connector advertises readOnlySql Pure extraction: no behavior change for the existing 7 drivers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(connectors): add MongoDB connector for issue #305 Add a read-only MongoDB connector that treats a database as a primary context source: collections map to tables and inferred top-level fields to columns. MongoDB is the first non-SQL source (readOnlySql: false), so ktx sql and metric compilation do not apply, but its collections flow through ingest, descriptions, and relationship discovery. - schema-inference: infer a flat column schema from the most recent sample_size documents (by _id desc, or order_by for non-ObjectId keys). Union BSON types per field, mark multi-type fields mixed (string), keep sub-documents/arrays as a single opaque json column, derive nullability from presence, treat _id as the primary key - connector: KtxMongoDbScanConnector behind an injectable client seam; strictly read-only (find/listCollections/estimatedDocumentCount only), no executeReadOnly; resolves env:/file: via resolveKtxConfigReference - core-only KtxMongoDbDialect and a live-database introspection adapter - wire the mongodb driver: driver union, dialect registry, driver registration (scopeConfigKey databases), mongodbConnectionSchema, connection-drivers, normalizeDriver, the live-database route, and the ktx setup picker. ktx sql is refused by the read-only SQL capability gate - tests: schema inference, connector snapshot via a fake client, dialect, driver-schema parsing, and the ktx sql rejection Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(integrations): document the MongoDB primary source Add a MongoDB section to the primary-sources reference: connection config (url, databases, enabled_tables, sample_size, order_by), mongodb+srv/TLS/ Atlas notes, the schema-inference explainer, a features matrix, and the non-SQL caveat. Update the frontmatter and connection field reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(connectors): address review blockers on the MongoDB connector - introspect: skip estimatedDocumentCount for views. The count command is rejected on a MongoDB view (CommandNotSupportedOnView), so counting a view aborted introspect for the whole connection; compute estimatedRows only for real collections, as ClickHouse does. - sl: refuse a semantic-layer query against a non-SQL connection instead of defaulting it to the Postgres dialect. compileLocalSlQuery (the shared CLI + MCP path) now rejects a driver with no SQL dialect via the new isSqlQueryableDriver authority, keeping MongoDB context-only per issue #305. - tests: cover input.tableScope and the empty-scope skip for the Mongo connector (the scan layer does not post-filter), the view no-count path, and the ktx sl query refusal for a mongodb connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * polish(mongodb): compute sampled nullCount and document sampling caveats Address the non-blocking review notes: - sampleColumn now counts null/absent values over the sampled window instead of returning nullCount: null, since the documents are already in hand - warn that a custom order_by must be indexed (an unindexed sort hits MongoDB's in-memory sort limit on large collections) in the connection schema and docs - note that sampled values for nested fields are stringified, not faithfully serialized, so the json opacity is deliberate Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(examples): add a MongoDB connector example A manual, container-backed example mirroring examples/postgres-historic: - docker-compose.yml + init/seed.js seed a representative dataset (nested documents, arrays, a Decimal128, a mixed-type field, a nullable field, an ObjectId reference, and a view) on first container start - scripts/smoke.sh + introspect-smoke.mjs assert the connector's inferred schema with no LLM credentials — the same introspection entry point ktx ingest's database-schema stage uses, including the view-no-count path - README.md documents the smoke and a full keyless ktx ingest run (claude-code LLM + managed sentence-transformers embeddings) Works with Docker Compose or podman compose. Verified end to end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: ignore examples/** in knip to fix dead-code false positives The MongoDB connector example files (examples/mongodb/init/seed.js and examples/mongodb/scripts/introspect-smoke.mjs) are used at runtime but were flagged as unused by knip. Add examples/** to the ignore array, matching the existing .context/** entry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114qQV8fJ5a5ME3XbMVRzbL * fix(mongodb): refuse non-SQL connections before SQL analysis `ktx sql` and the MCP sql_execution tool resolved a SQL-analysis dialect (falling back to Postgres for a non-SQL driver) and ran read-only validation before the connector capability gate refused the connection. For a MongoDB connection that spun up the parser/daemon and produced Postgres parser diagnostics instead of a clean non-SQL refusal. Route both entry points through a shared assertSqlQueryableConnection guard before dialect selection, mirroring compileLocalSlQuery. The federated duckdb path has no driver and is exempted at each call site. Add CLI and MCP regression tests asserting validation/connector work never starts for a MongoDB connection. * fix(mongodb): pass CI gates (dialect boundary, secrets, setup test) Three latent failures in the connector surfaced once CI ran on the branch: - connector.ts imported the concrete KtxMongoDbDialect, which the connector dialect-import boundary forbids. Route it through getDialectForDriver('mongodb') and widen inferKtxMongoCollectionColumns to the base KtxDialect (it only uses mapDataType/mapToDimensionType). - detect-secrets flagged a test ObjectId hex and the mongodb+srv example URL; annotate both with allowlist pragmas. - the "shows every supported database" setup test omitted the new MongoDB option. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Luca Martial <48870843+luca-martial@users.noreply.github.com> Co-authored-by: Luca Martial <lucamrtl@gmail.com> Co-authored-by: Andrey Avtomonov <andreybavt@gmail.com>
This commit is contained in:
parent
4f084186f1
commit
2afab61417
59 changed files with 1971 additions and 129 deletions
241
packages/cli/test/connectors/mongodb/connector.test.ts
Normal file
241
packages/cli/test/connectors/mongodb/connector.test.ts
Normal file
|
|
@ -0,0 +1,241 @@
|
|||
import { describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
isKtxMongoDbConnectionConfig,
|
||||
KtxMongoDbScanConnector,
|
||||
type KtxMongoClient,
|
||||
type KtxMongoClientFactory,
|
||||
type KtxMongoDbConnectionConfig,
|
||||
type KtxMongoListedCollection,
|
||||
} from '../../../src/connectors/mongodb/connector.js';
|
||||
import { createMongoDbLiveDatabaseIntrospection } from '../../../src/connectors/mongodb/live-database-introspection.js';
|
||||
import { executeProjectReadOnlySql } from '../../../src/context/connections/project-sql-executor.js';
|
||||
import { tableRefSet } from '../../../src/context/scan/table-ref.js';
|
||||
import type { KtxLocalProject } from '../../../src/context/project/project.js';
|
||||
import type { KtxMongoDocument } from '../../../src/connectors/mongodb/schema-inference.js';
|
||||
|
||||
function objectId(hex: string): unknown {
|
||||
return { _bsontype: 'ObjectId', toString: () => hex };
|
||||
}
|
||||
|
||||
const COLLECTIONS: Record<string, KtxMongoListedCollection[]> = {
|
||||
app: [
|
||||
{ name: 'users' },
|
||||
{ name: 'orders' },
|
||||
{ name: 'system.views' },
|
||||
],
|
||||
};
|
||||
|
||||
const DOCUMENTS: Record<string, KtxMongoDocument[]> = {
|
||||
users: [
|
||||
{ _id: objectId('a1'), email: 'a@x.com', age: 31, address: { city: 'NY' } },
|
||||
{ _id: objectId('a2'), email: 'b@x.com' },
|
||||
],
|
||||
orders: [{ _id: objectId('b1'), total: 9.99 }],
|
||||
};
|
||||
|
||||
function fakeClientFactory(): { factory: KtxMongoClientFactory; client: KtxMongoClient } {
|
||||
const client: KtxMongoClient = {
|
||||
listCollections: vi.fn(async (databaseName: string) => COLLECTIONS[databaseName] ?? []),
|
||||
estimatedDocumentCount: vi.fn(async (_databaseName: string, collectionName: string) =>
|
||||
collectionName === 'users' ? 2 : 1,
|
||||
),
|
||||
find: vi.fn(async (_databaseName: string, collectionName: string) => DOCUMENTS[collectionName] ?? []),
|
||||
ping: vi.fn(async () => undefined),
|
||||
close: vi.fn(async () => undefined),
|
||||
};
|
||||
return { factory: { create: vi.fn(() => client) }, client };
|
||||
}
|
||||
|
||||
function connector(connection: KtxMongoDbConnectionConfig, factory: KtxMongoClientFactory): KtxMongoDbScanConnector {
|
||||
return new KtxMongoDbScanConnector({ connectionId: 'mongo-prod', connection, clientFactory: factory });
|
||||
}
|
||||
|
||||
const baseConnection: KtxMongoDbConnectionConfig = {
|
||||
driver: 'mongodb',
|
||||
url: 'mongodb://localhost:27017/app',
|
||||
databases: ['app'],
|
||||
};
|
||||
|
||||
describe('isKtxMongoDbConnectionConfig', () => {
|
||||
it('matches only the mongodb driver', () => {
|
||||
expect(isKtxMongoDbConnectionConfig({ driver: 'mongodb' })).toBe(true);
|
||||
expect(isKtxMongoDbConnectionConfig({ driver: 'postgres' })).toBe(false);
|
||||
expect(isKtxMongoDbConnectionConfig(undefined)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('KtxMongoDbScanConnector capabilities', () => {
|
||||
it('advertises a non-SQL, sampling-capable source and exposes no SQL execution', () => {
|
||||
const { factory } = fakeClientFactory();
|
||||
const c = connector(baseConnection, factory);
|
||||
expect(c.driver).toBe('mongodb');
|
||||
expect(c.capabilities.readOnlySql).toBe(false);
|
||||
expect(c.capabilities.formalForeignKeys).toBe(false);
|
||||
expect(c.capabilities.tableSampling).toBe(true);
|
||||
expect(c.capabilities.columnSampling).toBe(true);
|
||||
expect(c.capabilities.nestedAnalysis).toBe(true);
|
||||
expect('executeReadOnly' in c).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('KtxMongoDbScanConnector construction', () => {
|
||||
it('requires a url', () => {
|
||||
const { factory } = fakeClientFactory();
|
||||
expect(() => connector({ driver: 'mongodb', databases: ['app'] }, factory)).toThrow(/requires connections\.mongo-prod\.url/);
|
||||
});
|
||||
|
||||
it('resolves env: url references and derives the database from the url path', () => {
|
||||
const { factory, client } = fakeClientFactory();
|
||||
const c = new KtxMongoDbScanConnector({
|
||||
connectionId: 'mongo-prod',
|
||||
connection: { driver: 'mongodb', url: 'env:MONGO_URL' },
|
||||
clientFactory: factory,
|
||||
env: { MONGO_URL: 'mongodb://localhost:27017/app' },
|
||||
});
|
||||
return c.introspect({ connectionId: 'mongo-prod', driver: 'mongodb' }, { runId: 't' }).then((snapshot) => {
|
||||
expect(snapshot.scope.schemas).toEqual(['app']);
|
||||
expect(client.listCollections).toHaveBeenCalledWith('app');
|
||||
});
|
||||
});
|
||||
|
||||
it('refuses a non-mongodb driver', () => {
|
||||
const { factory } = fakeClientFactory();
|
||||
expect(() => connector({ driver: 'postgres' } as KtxMongoDbConnectionConfig, factory)).toThrow(/cannot run driver "postgres"/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('KtxMongoDbScanConnector.introspect', () => {
|
||||
it('maps collections to tables, infers columns, and excludes system collections', async () => {
|
||||
const { factory } = fakeClientFactory();
|
||||
const snapshot = await connector(baseConnection, factory).introspect(
|
||||
{ connectionId: 'mongo-prod', driver: 'mongodb' },
|
||||
{ runId: 't' },
|
||||
);
|
||||
|
||||
expect(snapshot.driver).toBe('mongodb');
|
||||
expect(snapshot.tables.map((table) => table.name).sort()).toEqual(['orders', 'users']);
|
||||
|
||||
const users = snapshot.tables.find((table) => table.name === 'users')!;
|
||||
expect(users.db).toBe('app');
|
||||
expect(users.estimatedRows).toBe(2);
|
||||
expect(users.foreignKeys).toEqual([]);
|
||||
|
||||
const idColumn = users.columns.find((column) => column.name === '_id')!;
|
||||
expect(idColumn.primaryKey).toBe(true);
|
||||
expect(idColumn.nullable).toBe(false);
|
||||
|
||||
const email = users.columns.find((column) => column.name === 'email')!;
|
||||
expect(email.nullable).toBe(false); // present in every sampled document
|
||||
|
||||
const age = users.columns.find((column) => column.name === 'age')!;
|
||||
expect(age.nullable).toBe(true); // missing from the second document
|
||||
|
||||
const address = users.columns.find((column) => column.name === 'address')!;
|
||||
expect(address.normalizedType).toBe('json');
|
||||
});
|
||||
|
||||
it('honors the enabled_tables allowlist', async () => {
|
||||
const { factory } = fakeClientFactory();
|
||||
const snapshot = await connector(
|
||||
{ ...baseConnection, enabled_tables: ['app.users'] },
|
||||
factory,
|
||||
).introspect({ connectionId: 'mongo-prod', driver: 'mongodb' }, { runId: 't' });
|
||||
expect(snapshot.tables.map((table) => table.name)).toEqual(['users']);
|
||||
});
|
||||
|
||||
it('restricts introspection to input.tableScope (the scan layer does not post-filter)', async () => {
|
||||
const { factory } = fakeClientFactory();
|
||||
const snapshot = await connector(baseConnection, factory).introspect(
|
||||
{
|
||||
connectionId: 'mongo-prod',
|
||||
driver: 'mongodb',
|
||||
tableScope: tableRefSet([{ catalog: null, db: 'app', name: 'users' }]),
|
||||
},
|
||||
{ runId: 't' },
|
||||
);
|
||||
expect(snapshot.tables.map((table) => table.name)).toEqual(['users']);
|
||||
});
|
||||
|
||||
it('yields zero tables for a database whose scoped set is empty', async () => {
|
||||
const { factory } = fakeClientFactory();
|
||||
const snapshot = await connector(baseConnection, factory).introspect(
|
||||
{
|
||||
connectionId: 'mongo-prod',
|
||||
driver: 'mongodb',
|
||||
tableScope: tableRefSet([{ catalog: null, db: 'other', name: 'users' }]),
|
||||
},
|
||||
{ runId: 't' },
|
||||
);
|
||||
expect(snapshot.tables).toEqual([]);
|
||||
});
|
||||
|
||||
it('does not count documents on a view (estimatedDocumentCount fails on views)', async () => {
|
||||
const collections: KtxMongoListedCollection[] = [
|
||||
{ name: 'users' },
|
||||
{ name: 'active_users', type: 'view' },
|
||||
];
|
||||
const client: KtxMongoClient = {
|
||||
listCollections: vi.fn(async () => collections),
|
||||
// Real MongoDB rejects a count command on a view with CommandNotSupportedOnView.
|
||||
estimatedDocumentCount: vi.fn(async (_db: string, name: string) => {
|
||||
if (name === 'active_users') {
|
||||
throw new Error('CommandNotSupportedOnView: count is not supported on a view');
|
||||
}
|
||||
return 2;
|
||||
}),
|
||||
find: vi.fn(async (_db: string, name: string) => DOCUMENTS[name] ?? DOCUMENTS.users!),
|
||||
ping: vi.fn(async () => undefined),
|
||||
close: vi.fn(async () => undefined),
|
||||
};
|
||||
const snapshot = await connector(baseConnection, { create: vi.fn(() => client) }).introspect(
|
||||
{ connectionId: 'mongo-prod', driver: 'mongodb' },
|
||||
{ runId: 't' },
|
||||
);
|
||||
const view = snapshot.tables.find((table) => table.name === 'active_users')!;
|
||||
expect(view.kind).toBe('view');
|
||||
expect(view.estimatedRows).toBeNull();
|
||||
expect(client.estimatedDocumentCount).not.toHaveBeenCalledWith('app', 'active_users');
|
||||
});
|
||||
});
|
||||
|
||||
describe('KtxMongoDbScanConnector sampling', () => {
|
||||
it('samples a column, flattening nested values and counting nulls over the window', async () => {
|
||||
const { factory } = fakeClientFactory();
|
||||
const result = await connector(baseConnection, factory).sampleColumn(
|
||||
{ connectionId: 'mongo-prod', table: { catalog: null, db: 'app', name: 'users' }, column: 'address', limit: 10 },
|
||||
{ runId: 't' },
|
||||
);
|
||||
expect(result.values).toEqual([JSON.stringify({ city: 'NY' })]);
|
||||
// address is present in the first sampled document and absent from the second
|
||||
expect(result.nullCount).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('createMongoDbLiveDatabaseIntrospection', () => {
|
||||
it('extracts a schema through the live-database port', async () => {
|
||||
const { factory } = fakeClientFactory();
|
||||
const port = createMongoDbLiveDatabaseIntrospection({
|
||||
connections: {
|
||||
'mongo-prod': { driver: 'mongodb', url: 'mongodb://localhost:27017/app', databases: ['app'] },
|
||||
},
|
||||
clientFactory: factory,
|
||||
});
|
||||
const snapshot = await port.extractSchema('mongo-prod');
|
||||
expect(snapshot.driver).toBe('mongodb');
|
||||
expect(snapshot.tables.length).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('ktx sql against a MongoDB connection', () => {
|
||||
it('is rejected by the read-only SQL capability gate', async () => {
|
||||
const { factory } = fakeClientFactory();
|
||||
const project = { config: { connections: {} }, projectDir: '/tmp' } as unknown as KtxLocalProject;
|
||||
await expect(
|
||||
executeProjectReadOnlySql({
|
||||
project,
|
||||
input: { connectionId: 'mongo-prod', connection: undefined, sql: 'SELECT 1', maxRows: 1 },
|
||||
createConnector: () => connector(baseConnection, factory),
|
||||
}),
|
||||
).rejects.toThrow(/does not support read-only SQL execution/);
|
||||
});
|
||||
});
|
||||
33
packages/cli/test/connectors/mongodb/dialect.test.ts
Normal file
33
packages/cli/test/connectors/mongodb/dialect.test.ts
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
import { describe, expect, it } from 'vitest';
|
||||
import { KtxMongoDbDialect } from '../../../src/connectors/mongodb/dialect.js';
|
||||
import { getDialectForDriver, getSqlDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
|
||||
const dialect = new KtxMongoDbDialect();
|
||||
|
||||
describe('KtxMongoDbDialect', () => {
|
||||
it('formats and parses db.collection display refs', () => {
|
||||
expect(dialect.formatDisplayRef({ catalog: null, db: 'app', name: 'users' })).toBe('app.users');
|
||||
expect(dialect.parseDisplayRef('app.users')).toEqual({ catalog: null, db: 'app', name: 'users' });
|
||||
expect(dialect.columnDisplayTablePartCount()).toBe(2);
|
||||
});
|
||||
|
||||
it('maps nested types to opaque json and scalars to dimension types', () => {
|
||||
expect(dialect.mapDataType('object')).toBe('json');
|
||||
expect(dialect.mapDataType('array')).toBe('json');
|
||||
expect(dialect.mapDataType('objectId')).toBe('objectid');
|
||||
expect(dialect.mapToDimensionType('date')).toBe('time');
|
||||
expect(dialect.mapToDimensionType('long')).toBe('number');
|
||||
expect(dialect.mapToDimensionType('mixed')).toBe('string');
|
||||
expect(dialect.mapToDimensionType('object')).toBe('string');
|
||||
});
|
||||
});
|
||||
|
||||
describe('dialect registry', () => {
|
||||
it('resolves a core dialect for mongodb', () => {
|
||||
expect(getDialectForDriver('mongodb').type).toBe('mongodb');
|
||||
});
|
||||
|
||||
it('refuses a SQL dialect for mongodb', () => {
|
||||
expect(() => getSqlDialectForDriver('mongodb')).toThrow(/no SQL dialect/);
|
||||
});
|
||||
});
|
||||
103
packages/cli/test/connectors/mongodb/schema-inference.test.ts
Normal file
103
packages/cli/test/connectors/mongodb/schema-inference.test.ts
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
import { describe, expect, it } from 'vitest';
|
||||
import { KtxMongoDbDialect } from '../../../src/connectors/mongodb/dialect.js';
|
||||
import {
|
||||
bsonTypeOf,
|
||||
inferKtxMongoCollectionColumns,
|
||||
type KtxMongoDocument,
|
||||
} from '../../../src/connectors/mongodb/schema-inference.js';
|
||||
|
||||
const dialect = new KtxMongoDbDialect();
|
||||
|
||||
function objectId(): unknown {
|
||||
return { _bsontype: 'ObjectId', toString: () => '64b7f0c2a1b2c3d4e5f60718' }; // pragma: allowlist secret
|
||||
}
|
||||
|
||||
function decimal128(value: string): unknown {
|
||||
return { _bsontype: 'Decimal128', toString: () => value };
|
||||
}
|
||||
|
||||
function infer(documents: KtxMongoDocument[]) {
|
||||
const columns = inferKtxMongoCollectionColumns(documents, dialect);
|
||||
return new Map(columns.map((column) => [column.name, column]));
|
||||
}
|
||||
|
||||
describe('bsonTypeOf', () => {
|
||||
it('maps JS and BSON runtime values to canonical type names', () => {
|
||||
expect(bsonTypeOf(objectId())).toBe('objectId');
|
||||
expect(bsonTypeOf('hi')).toBe('string');
|
||||
expect(bsonTypeOf(7)).toBe('int');
|
||||
expect(bsonTypeOf(7.5)).toBe('double');
|
||||
expect(bsonTypeOf(true)).toBe('bool');
|
||||
expect(bsonTypeOf(new Date())).toBe('date');
|
||||
expect(bsonTypeOf(decimal128('1.5'))).toBe('decimal');
|
||||
expect(bsonTypeOf({ city: 'NY' })).toBe('object');
|
||||
expect(bsonTypeOf([1, 2])).toBe('array');
|
||||
expect(bsonTypeOf(null)).toBe('null');
|
||||
});
|
||||
});
|
||||
|
||||
describe('inferKtxMongoCollectionColumns', () => {
|
||||
it('treats _id as the non-nullable primary key', () => {
|
||||
const columns = infer([{ _id: objectId(), name: 'a' }]);
|
||||
const id = columns.get('_id')!;
|
||||
expect(id.primaryKey).toBe(true);
|
||||
expect(id.nullable).toBe(false);
|
||||
expect(id.dimensionType).toBe('string');
|
||||
expect(id.normalizedType).toBe('objectid');
|
||||
});
|
||||
|
||||
it('derives nullability from field presence and observed nulls', () => {
|
||||
const columns = infer([
|
||||
{ _id: objectId(), email: 'a@x.com', deleted_at: null },
|
||||
{ _id: objectId(), email: 'b@x.com' },
|
||||
]);
|
||||
// present in every document, never null -> not nullable
|
||||
expect(columns.get('email')!.nullable).toBe(false);
|
||||
// missing in one document and null in another -> nullable
|
||||
expect(columns.get('deleted_at')!.nullable).toBe(true);
|
||||
});
|
||||
|
||||
it('maps scalar BSON types to dimension types', () => {
|
||||
const columns = infer([
|
||||
{ _id: objectId(), age: 30, score: 9.5, active: true, created: new Date(), balance: decimal128('10.00') },
|
||||
]);
|
||||
expect(columns.get('age')!.dimensionType).toBe('number');
|
||||
expect(columns.get('score')!.dimensionType).toBe('number');
|
||||
expect(columns.get('active')!.dimensionType).toBe('boolean');
|
||||
expect(columns.get('created')!.dimensionType).toBe('time');
|
||||
expect(columns.get('balance')!.dimensionType).toBe('number');
|
||||
});
|
||||
|
||||
it('marks a field seen with more than one type as mixed and treats it as a string', () => {
|
||||
const columns = infer([
|
||||
{ _id: objectId(), ref: 'abc' },
|
||||
{ _id: objectId(), ref: 123 },
|
||||
]);
|
||||
const ref = columns.get('ref')!;
|
||||
expect(ref.nativeType).toBe('mixed');
|
||||
expect(ref.normalizedType).toBe('mixed');
|
||||
expect(ref.dimensionType).toBe('string');
|
||||
});
|
||||
|
||||
it('keeps sub-documents and arrays as a single opaque json column', () => {
|
||||
const columns = infer([
|
||||
{ _id: objectId(), address: { city: 'NY', zip: '10001' }, tags: ['a', 'b'] },
|
||||
]);
|
||||
const address = columns.get('address')!;
|
||||
expect(address.nativeType).toBe('object');
|
||||
expect(address.normalizedType).toBe('json');
|
||||
expect(address.dimensionType).toBe('string');
|
||||
|
||||
const tags = columns.get('tags')!;
|
||||
expect(tags.nativeType).toBe('array');
|
||||
expect(tags.normalizedType).toBe('json');
|
||||
});
|
||||
|
||||
it('preserves first-seen field order', () => {
|
||||
const columns = inferKtxMongoCollectionColumns(
|
||||
[{ _id: objectId(), b: 1, a: 2 }],
|
||||
dialect,
|
||||
);
|
||||
expect(columns.map((column) => column.name)).toEqual(['_id', 'b', 'a']);
|
||||
});
|
||||
});
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
import { describe, expect, it } from 'vitest';
|
||||
import { getDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
import { getDialectForDriver, getSqlDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
import type { KtxConnectionDriver, KtxTableRef } from '../../../src/context/scan/types.js';
|
||||
|
||||
interface DialectFixture {
|
||||
|
|
@ -248,8 +248,8 @@ const fixtures: DialectFixture[] = [
|
|||
];
|
||||
|
||||
describe('getDialectForDriver', () => {
|
||||
it.each(fixtures)('returns a full KtxDialect for $driver', (fixture) => {
|
||||
const dialect = getDialectForDriver(fixture.driver);
|
||||
it.each(fixtures)('returns a full KtxSqlDialect for $driver', (fixture) => {
|
||||
const dialect = getSqlDialectForDriver(fixture.driver);
|
||||
const column = dialect.quoteIdentifier('status');
|
||||
|
||||
expect(dialect.type).toBe(fixture.driver);
|
||||
|
|
@ -305,12 +305,12 @@ describe('getDialectForDriver', () => {
|
|||
|
||||
it('throws with a supported-driver list for unknown drivers', () => {
|
||||
expect(() => getDialectForDriver('oracle')).toThrow(
|
||||
'Unsupported warehouse driver "oracle". Supported drivers: bigquery, clickhouse, mysql, postgres, sqlite, snowflake, sqlserver',
|
||||
'Unsupported driver "oracle". Supported drivers: bigquery, clickhouse, mongodb, mysql, postgres, snowflake, sqlite, sqlserver',
|
||||
);
|
||||
});
|
||||
|
||||
it('rejects legacy driver aliases', () => {
|
||||
expect(() => getDialectForDriver('postgresql')).toThrow('Unsupported warehouse driver "postgresql"');
|
||||
expect(() => getDialectForDriver('sqlite3')).toThrow('Unsupported warehouse driver "sqlite3"');
|
||||
expect(() => getDialectForDriver('postgresql')).toThrow('Unsupported driver "postgresql"');
|
||||
expect(() => getDialectForDriver('sqlite3')).toThrow('Unsupported driver "sqlite3"');
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -22,6 +22,11 @@ const connectionFixtures: Record<KtxConnectionDriver, FixtureFactory> = {
|
|||
schemas: ['public'],
|
||||
}),
|
||||
sqlite: () => ({ driver: 'sqlite', path: 'warehouse.db' }),
|
||||
mongodb: () => ({
|
||||
driver: 'mongodb',
|
||||
url: 'mongodb://localhost:27017/app',
|
||||
databases: ['app'],
|
||||
}),
|
||||
mysql: () => ({
|
||||
driver: 'mysql',
|
||||
host: 'localhost',
|
||||
|
|
@ -96,6 +101,7 @@ describe('driverRegistrations', () => {
|
|||
expect(listSupportedDrivers()).toEqual([
|
||||
'bigquery',
|
||||
'clickhouse',
|
||||
'mongodb',
|
||||
'mysql',
|
||||
'postgres',
|
||||
'snowflake',
|
||||
|
|
|
|||
|
|
@ -282,6 +282,33 @@ describe('createLocalProjectMcpContextPorts', () => {
|
|||
expect(createConnector).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('refuses sql_execution against a non-SQL (MongoDB) connection before SQL analysis', async () => {
|
||||
const project = await initKtxProject({ projectDir: tempDir });
|
||||
project.config.connections.mongo = { driver: 'mongodb', url: 'mongodb://localhost:27017/app' };
|
||||
const createConnector = vi.fn(async () => testConnector());
|
||||
const sqlAnalysis = {
|
||||
analyzeForFingerprint: vi.fn(),
|
||||
analyzeBatch: vi.fn(),
|
||||
validateReadOnly: vi.fn(async () => ({ ok: true, error: null })),
|
||||
};
|
||||
const ports = createLocalProjectMcpContextPorts(project, {
|
||||
sqlAnalysis,
|
||||
localScan: { createConnector },
|
||||
embeddingService: null,
|
||||
});
|
||||
|
||||
const execution = ports.sqlExecution?.execute({
|
||||
connectionId: 'mongo',
|
||||
sql: 'select 1',
|
||||
maxRows: 5,
|
||||
});
|
||||
await expect(execution).rejects.toBeInstanceOf(KtxExpectedError);
|
||||
await expect(execution).rejects.toThrow("non-SQL driver 'mongodb'");
|
||||
// Refused before the parser dialect is chosen and before any connector is built.
|
||||
expect(sqlAnalysis.validateReadOnly).not.toHaveBeenCalled();
|
||||
expect(createConnector).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('emits sql_execution progress stages from local MCP ports', async () => {
|
||||
const project = await initKtxProject({ projectDir: tempDir });
|
||||
project.config.connections.warehouse = {
|
||||
|
|
|
|||
|
|
@ -28,6 +28,28 @@ describe('connectionConfigSchema (driver discriminated union)', () => {
|
|||
});
|
||||
});
|
||||
|
||||
it('parses a mongodb connection with sampling fields', () => {
|
||||
const parsed = connectionConfigSchema.parse({
|
||||
driver: 'mongodb',
|
||||
url: 'env:MONGO_URL',
|
||||
databases: ['app'],
|
||||
enabled_tables: ['app.users'],
|
||||
sample_size: 500,
|
||||
order_by: 'createdAt',
|
||||
});
|
||||
expect(parsed).toMatchObject({
|
||||
driver: 'mongodb',
|
||||
url: 'env:MONGO_URL',
|
||||
databases: ['app'],
|
||||
sample_size: 500,
|
||||
order_by: 'createdAt',
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects a mongodb connection without a url', () => {
|
||||
expect(() => connectionConfigSchema.parse({ driver: 'mongodb', databases: ['app'] })).toThrow();
|
||||
});
|
||||
|
||||
it('rejects an unknown driver', () => {
|
||||
expect(() => connectionConfigSchema.parse({ driver: 'nope', url: 'x' })).toThrow();
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import Database from 'better-sqlite3';
|
||||
import { join } from 'node:path';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { getDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
import { getSqlDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
import { snapshotToKtxEnrichedSchema } from '../../../src/context/scan/local-enrichment.js';
|
||||
import { loadKtxRelationshipBenchmarkFixture, maskKtxRelationshipBenchmarkSnapshot } from '../../../src/context/scan/relationship-benchmarks.js';
|
||||
import { discoverKtxCompositeRelationships } from '../../../src/context/scan/relationship-composite-candidates.js';
|
||||
|
|
@ -42,7 +42,8 @@ describe('composite relationship discovery detector', () => {
|
|||
const executor = new TestSqliteExecutor(fixture.dataPath ?? '');
|
||||
const profiles = await profileKtxRelationshipSchema({
|
||||
connectionId: snapshot.connectionId,
|
||||
dialect: getDialectForDriver(snapshot.driver),
|
||||
driver: snapshot.driver,
|
||||
dialect: getSqlDialectForDriver(snapshot.driver),
|
||||
schema,
|
||||
executor,
|
||||
ctx: { runId: 'test:composite-profile' },
|
||||
|
|
@ -50,7 +51,7 @@ describe('composite relationship discovery detector', () => {
|
|||
|
||||
const result = await discoverKtxCompositeRelationships({
|
||||
connectionId: snapshot.connectionId,
|
||||
dialect: getDialectForDriver(snapshot.driver),
|
||||
dialect: getSqlDialectForDriver(snapshot.driver),
|
||||
schema,
|
||||
profiles,
|
||||
executor,
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import Database from 'better-sqlite3';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import type { KtxLlmRuntimePort } from '../../../src/context/llm/runtime-port.js';
|
||||
import { getDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
import { getSqlDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
import { buildDefaultKtxProjectConfig } from '../../../src/context/project/config.js';
|
||||
import { snapshotToKtxEnrichedSchema } from '../../../src/context/scan/local-enrichment.js';
|
||||
import {
|
||||
|
|
@ -311,7 +311,7 @@ describe('production relationship discovery', () => {
|
|||
|
||||
const result = await discoverKtxRelationships({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
connector: connector(executor),
|
||||
schema: snapshotToKtxEnrichedSchema(snapshot()),
|
||||
context: { runId: 'relationship-run-1' },
|
||||
|
|
@ -350,7 +350,7 @@ describe('production relationship discovery', () => {
|
|||
const schema = naturalKeySnapshot();
|
||||
const result = await discoverKtxRelationships({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
connector: {
|
||||
...connector(executor),
|
||||
introspect: async () => schema,
|
||||
|
|
@ -400,7 +400,7 @@ describe('production relationship discovery', () => {
|
|||
|
||||
const result = await discoverKtxRelationships({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
connector: {
|
||||
...connector(executor),
|
||||
introspect: async () => sourceSnapshot,
|
||||
|
|
@ -433,7 +433,7 @@ describe('production relationship discovery', () => {
|
|||
it('keeps candidates review-only when read-only SQL is unavailable', async () => {
|
||||
const result = await discoverKtxRelationships({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
connector: connector(null),
|
||||
schema: snapshotToKtxEnrichedSchema(snapshot()),
|
||||
context: { runId: 'relationship-run-no-sql' },
|
||||
|
|
@ -459,7 +459,7 @@ describe('production relationship discovery', () => {
|
|||
const sourceSnapshot = declaredForeignKeySnapshot();
|
||||
const result = await discoverKtxRelationships({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
connector: connector(null),
|
||||
schema: snapshotToKtxEnrichedSchema(sourceSnapshot),
|
||||
context: { runId: 'formal-metadata-no-sql' },
|
||||
|
|
@ -506,7 +506,7 @@ describe('production relationship discovery', () => {
|
|||
|
||||
const result = await discoverKtxRelationships({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
connector: connector(executor),
|
||||
schema: snapshotToKtxEnrichedSchema(llmOnlyRelationshipSnapshot()),
|
||||
context: { runId: 'llm-relationship-orchestrator' },
|
||||
|
|
@ -546,7 +546,7 @@ describe('production relationship discovery', () => {
|
|||
|
||||
const result = await discoverKtxRelationships({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
connector: connector(executor),
|
||||
schema: snapshotToKtxEnrichedSchema(snapshot()),
|
||||
context: { runId: 'configured-thresholds' },
|
||||
|
|
@ -607,7 +607,7 @@ describe('production relationship discovery', () => {
|
|||
|
||||
const result = await discoverKtxRelationships({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
connector: {
|
||||
...connector(executor),
|
||||
introspect: async () => richSnapshot,
|
||||
|
|
@ -663,7 +663,7 @@ describe('production relationship discovery', () => {
|
|||
|
||||
const result = await discoverKtxRelationships({
|
||||
connectionId: maskedSnapshot.connectionId,
|
||||
dialect: getDialectForDriver(maskedSnapshot.driver),
|
||||
dialect: getSqlDialectForDriver(maskedSnapshot.driver),
|
||||
connector: testConnector,
|
||||
schema: snapshotToKtxEnrichedSchema(maskedSnapshot, new Map()),
|
||||
context: { runId: 'test:production-composite' },
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import { readFile } from 'node:fs/promises';
|
|||
import { join } from 'node:path';
|
||||
import Database from 'better-sqlite3';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import { getDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
import { getSqlDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
import type { KtxEnrichedColumn, KtxEnrichedSchema, KtxEnrichedTable } from '../../../src/context/scan/enrichment-types.js';
|
||||
import { snapshotToKtxEnrichedSchema } from '../../../src/context/scan/local-enrichment.js';
|
||||
import { loadKtxRelationshipBenchmarkFixture, maskKtxRelationshipBenchmarkSnapshot } from '../../../src/context/scan/relationship-benchmarks.js';
|
||||
|
|
@ -121,7 +121,8 @@ describe('relationship profiling', () => {
|
|||
|
||||
const result = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: schema([
|
||||
table('accounts', [
|
||||
column('accounts', 'id', { primaryKey: false, nullable: false }),
|
||||
|
|
@ -183,7 +184,8 @@ describe('relationship profiling', () => {
|
|||
|
||||
const result = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: schema([
|
||||
table('accounts', [
|
||||
column('accounts', 'id', { nullable: false }),
|
||||
|
|
@ -226,7 +228,8 @@ describe('relationship profiling', () => {
|
|||
|
||||
const profiles = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: schema([
|
||||
table('accounts', [
|
||||
column('accounts', 'id', { nullable: false }),
|
||||
|
|
@ -277,7 +280,8 @@ describe('relationship profiling', () => {
|
|||
|
||||
const first = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: relationshipSchema,
|
||||
executor,
|
||||
ctx: { runId: 'profile-cache-run' },
|
||||
|
|
@ -285,7 +289,8 @@ describe('relationship profiling', () => {
|
|||
});
|
||||
const second = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: relationshipSchema,
|
||||
executor,
|
||||
ctx: { runId: 'profile-cache-run' },
|
||||
|
|
@ -293,7 +298,8 @@ describe('relationship profiling', () => {
|
|||
});
|
||||
const third = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: relationshipSchema,
|
||||
executor,
|
||||
ctx: { runId: 'profile-cache-fresh-run' },
|
||||
|
|
@ -322,7 +328,8 @@ describe('relationship profiling', () => {
|
|||
try {
|
||||
const result = await profileKtxRelationshipSchema({
|
||||
connectionId: fixture.snapshot.connectionId,
|
||||
dialect: getDialectForDriver(fixture.snapshot.driver),
|
||||
driver: fixture.snapshot.driver,
|
||||
dialect: getSqlDialectForDriver(fixture.snapshot.driver),
|
||||
schema: snapshotToKtxEnrichedSchema(maskedSnapshot, new Map()),
|
||||
executor: scaleExecutor,
|
||||
ctx: { runId: 'scale-stress-profile-query-count' },
|
||||
|
|
@ -367,7 +374,8 @@ describe('relationship profiling', () => {
|
|||
|
||||
await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: schemaWithTables(['accounts', 'orders', 'payments', 'refunds']),
|
||||
executor,
|
||||
ctx: { runId: 'profile-concurrency' },
|
||||
|
|
@ -403,7 +411,8 @@ describe('relationship profiling', () => {
|
|||
|
||||
const result = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: schemaWithTables(['accounts', 'orders']),
|
||||
executor,
|
||||
ctx: { runId: 'profile-error-isolated' },
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
import Database from 'better-sqlite3';
|
||||
import { afterEach, describe, expect, it } from 'vitest';
|
||||
import { getDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
import { getSqlDialectForDriver } from '../../../src/context/connections/dialects.js';
|
||||
import type { KtxEnrichedColumn, KtxEnrichedSchema, KtxEnrichedTable } from '../../../src/context/scan/enrichment-types.js';
|
||||
import { generateKtxRelationshipDiscoveryCandidates } from '../../../src/context/scan/relationship-candidates.js';
|
||||
import type { KtxRelationshipProfileArtifact } from '../../../src/context/scan/relationship-profiling.js';
|
||||
|
|
@ -102,7 +102,8 @@ describe('relationship validation', () => {
|
|||
const testSchema = schema();
|
||||
const profiles = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: testSchema,
|
||||
executor,
|
||||
ctx: { runId: 'validate-test' },
|
||||
|
|
@ -113,7 +114,7 @@ describe('relationship validation', () => {
|
|||
|
||||
const validated = await validateKtxRelationshipDiscoveryCandidates({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
candidates,
|
||||
profiles,
|
||||
executor,
|
||||
|
|
@ -151,7 +152,8 @@ describe('relationship validation', () => {
|
|||
const testSchema = schema();
|
||||
const profiles = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: testSchema,
|
||||
executor,
|
||||
ctx: { runId: 'validate-test' },
|
||||
|
|
@ -162,7 +164,7 @@ describe('relationship validation', () => {
|
|||
|
||||
const validated = await validateKtxRelationshipDiscoveryCandidates({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
candidates,
|
||||
profiles,
|
||||
executor,
|
||||
|
|
@ -201,7 +203,8 @@ describe('relationship validation', () => {
|
|||
const testSchema = schema();
|
||||
const profiles = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: testSchema,
|
||||
executor,
|
||||
ctx: { runId: 'validate-budget-profile' },
|
||||
|
|
@ -214,7 +217,7 @@ describe('relationship validation', () => {
|
|||
|
||||
const validated = await validateKtxRelationshipDiscoveryCandidates({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
candidates,
|
||||
profiles,
|
||||
executor,
|
||||
|
|
@ -256,7 +259,8 @@ describe('relationship validation', () => {
|
|||
]);
|
||||
const profiles = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: testSchema,
|
||||
executor,
|
||||
ctx: { runId: 'validate-zero-budget-profile' },
|
||||
|
|
@ -266,7 +270,7 @@ describe('relationship validation', () => {
|
|||
|
||||
const validated = await validateKtxRelationshipDiscoveryCandidates({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
candidates,
|
||||
profiles,
|
||||
executor,
|
||||
|
|
@ -303,7 +307,8 @@ describe('relationship validation', () => {
|
|||
]);
|
||||
const profiles = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: testSchema,
|
||||
executor,
|
||||
ctx: { runId: 'llm-rejected-validation' },
|
||||
|
|
@ -332,7 +337,7 @@ describe('relationship validation', () => {
|
|||
|
||||
const [validated] = await validateKtxRelationshipDiscoveryCandidates({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
candidates: [llmCandidate],
|
||||
profiles,
|
||||
executor,
|
||||
|
|
@ -377,7 +382,8 @@ describe('relationship validation', () => {
|
|||
]);
|
||||
const profiles = await profileKtxRelationshipSchema({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
driver: 'sqlite',
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
schema: testSchema,
|
||||
executor,
|
||||
ctx: { runId: 'validation-concurrency-profile' },
|
||||
|
|
@ -386,7 +392,7 @@ describe('relationship validation', () => {
|
|||
|
||||
await validateKtxRelationshipDiscoveryCandidates({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
candidates,
|
||||
profiles,
|
||||
executor: throttled,
|
||||
|
|
@ -478,7 +484,7 @@ describe('relationship validation', () => {
|
|||
|
||||
const [validated] = await validateKtxRelationshipDiscoveryCandidates({
|
||||
connectionId: 'warehouse',
|
||||
dialect: getDialectForDriver('sqlite'),
|
||||
dialect: getSqlDialectForDriver('sqlite'),
|
||||
candidates: [candidate],
|
||||
profiles,
|
||||
executor,
|
||||
|
|
|
|||
|
|
@ -67,6 +67,18 @@ grain: []
|
|||
await rm(tempDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('refuses a non-SQL (context-only) connection instead of compiling it as Postgres', async () => {
|
||||
project.config.connections['mongo-prod'] = { driver: 'mongodb', url: 'mongodb://localhost:27017/app' };
|
||||
await expect(
|
||||
compileLocalSlQuery(project, {
|
||||
connectionId: 'mongo-prod',
|
||||
query: { measures: ['orders.order_count'], dimensions: ['orders.status'], limit: 25 },
|
||||
compute,
|
||||
}),
|
||||
).rejects.toThrow(/non-SQL driver 'mongodb'|require a SQL warehouse connection/);
|
||||
expect(compute.query).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('compiles a local semantic-layer query with computable sources only', async () => {
|
||||
const result = await compileLocalSlQuery(project, {
|
||||
connectionId: 'warehouse',
|
||||
|
|
|
|||
|
|
@ -243,6 +243,7 @@ describe('setup databases step', () => {
|
|||
{ value: 'mysql', label: 'MySQL' },
|
||||
{ value: 'clickhouse', label: 'ClickHouse' },
|
||||
{ value: 'sqlserver', label: 'SQL Server' },
|
||||
{ value: 'mongodb', label: 'MongoDB' },
|
||||
{ value: 'sqlite', label: 'SQLite' },
|
||||
],
|
||||
required: true,
|
||||
|
|
|
|||
|
|
@ -348,6 +348,40 @@ describe('runKtxSql', () => {
|
|||
expect(io.stderr()).toContain('does not support read-only SQL execution.');
|
||||
});
|
||||
|
||||
it('refuses a non-SQL (MongoDB) connection before invoking SQL analysis', async () => {
|
||||
const projectDir = join(tempDir, 'project');
|
||||
await initKtxProject({ projectDir });
|
||||
await writeConnections(projectDir, { mongo: { driver: 'mongodb', url: 'mongodb://localhost:27017/app' } });
|
||||
const sqlAnalysis = makeSqlAnalysis({ ok: true, error: null });
|
||||
const createSqlAnalysis = vi.fn(() => sqlAnalysis);
|
||||
const createScanConnector = vi.fn(async () => makeConnector());
|
||||
const io = makeIo();
|
||||
|
||||
await expect(
|
||||
runKtxSql(
|
||||
{
|
||||
command: 'execute',
|
||||
projectDir,
|
||||
connectionId: 'mongo',
|
||||
sql: 'select 1',
|
||||
maxRows: 1000,
|
||||
output: 'pretty',
|
||||
json: false,
|
||||
cliVersion: '0.0.0-test',
|
||||
},
|
||||
io.io,
|
||||
{ createSqlAnalysis, createScanConnector },
|
||||
),
|
||||
).resolves.toBe(1);
|
||||
|
||||
// The non-SQL boundary is enforced before any SQL parser/daemon work, so a
|
||||
// MongoDB connection never reaches dialect selection or read-only validation.
|
||||
expect(createSqlAnalysis).not.toHaveBeenCalled();
|
||||
expect(sqlAnalysis.validateReadOnly).not.toHaveBeenCalled();
|
||||
expect(createScanConnector).not.toHaveBeenCalled();
|
||||
expect(io.stderr()).toContain("non-SQL driver 'mongodb'");
|
||||
});
|
||||
|
||||
it('routes _ktx_federated through the shared federated executor', async () => {
|
||||
const projectDir = join(tempDir, 'project');
|
||||
await initKtxProject({ projectDir });
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue