ktx/examples/postgres-historic/README.md

Postgres Historic SQL Example

This example is a manual smoke for Postgres historic-SQL ingest through pg_stat_statements. It starts Postgres 14 with the extension preloaded, generates query workload under separate users, runs klo setup with --enable-historic-sql, and verifies three local ingest runs:

• first run creates a fresh PGSS baseline
• second run emits only positive deltas
• reset run treats pg_stat_statements_reset() as a fresh baseline

Prerequisites

• Docker with Compose v2
• Node and pnpm matching the KLO workspace
• python-service/.venv already created, or KLO_SQL_ANALYSIS_URL pointing at a running service that exposes /api/sql/analyze-for-fingerprint

Run

From the KLO repository root:

examples/postgres-historic/scripts/smoke.sh

The smoke creates a temporary KLO project, starts Postgres on 127.0.0.1:55432, and uses this connection URL:

postgresql://klo_reader:klo_reader@127.0.0.1:55432/analytics # pragma: allowlist secret

Set KLO_POSTGRES_HISTORIC_KEEP_DOCKER=1 to leave the container running after the script exits.

The smoke validates the historic-SQL raw snapshot path without requiring LLM credentials. It uses KLO's local stage-only ingest API after klo setup so the PGSS baseline and delta behavior can be checked independently from curation.

Manual Commands

Start Postgres and generate the base workload:

docker compose -f examples/postgres-historic/docker-compose.yml up -d --wait
examples/postgres-historic/scripts/generate-workload.sh base

Create a project and enable historic SQL:

export WAREHOUSE_DATABASE_URL=postgresql://klo_reader:klo_reader@127.0.0.1:55432/analytics # pragma: allowlist secret
pnpm --filter @klo/cli run build
node packages/cli/dist/bin.js --project-dir /tmp/klo-postgres-historic setup \
  --new \
  --skip-agents \
  --skip-llm \
  --skip-embeddings \
  --skip-sources \
  --database postgres \
  --new-database-connection-id warehouse \
  --database-url env:WAREHOUSE_DATABASE_URL \
  --database-schema public \
  --enable-historic-sql \
  --historic-sql-min-calls 2 \
  --yes \
  --no-input

Readiness check

pnpm run klo -- dev doctor --project-dir /tmp/klo-postgres-historic --no-input

The installed CLI form is klo dev doctor --project-dir /tmp/klo-postgres-historic --no-input. Expected output includes PASS Postgres Historic SQL (warehouse) when pg_stat_statements is installed, pg_read_all_stats is granted, tracking is enabled, and pg_stat_statements.max is at least 5000.

Run local historic-SQL ingest:

node packages/cli/dist/bin.js --project-dir /tmp/klo-postgres-historic dev ingest run \
  --connection-id warehouse \
  --adapter historic-sql \
  --plain \
  --no-input

The full dev ingest run path also runs curation work units, so it requires a configured LLM provider.

Inspect the latest manifest:

find /tmp/klo-postgres-historic/raw-sources/warehouse/historic-sql -name manifest.json | sort | tail -n 1

The manifest should have dialect: "postgres", degraded: true, baselineFirstRun: true on the first run, and populated pgServerVersion and statsResetAt.

Troubleshooting

• Missing extension: confirm shared_preload_libraries=pg_stat_statements and CREATE EXTENSION pg_stat_statements; both happened in the analytics database.
• Missing grants: confirm GRANT pg_read_all_stats TO klo_reader;.
• Empty templates: rerun scripts/generate-workload.sh base and keep --historic-sql-min-calls 2 for the smoke.
• SQL-analysis failures: set KLO_SQL_ANALYSIS_URL to the running service URL or create python-service/.venv before running scripts/smoke.sh.