Add full pipeline test that generates a real PDF, processes it through
the entire pipeline, and verifies knowledge lands in FalkorDB:
- Create test PDF generator using pdf-lib (2-page doc about Acme Corp)
- Add testFullPipeline() to integration tests with store verification
- Fix FalkorDB client connect() — createClient returns unconnected client
in both TriplesStore and TriplesQuery classes
Results: PDF decoded (2 pages) → chunked (2 chunks) → extracted
(4 relationships) → 16 triples stored in FalkorDB including:
alice-johnson → is-a-senior-engineer → acme-corporation
cloudsync → uses-aws-for-hosting → amazon-web-services
provenance: pages → prov:wasDerivedFrom → source document
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two bugs found during end-to-end testing:
1. FlowProcessor never restarted flows when config changed — it only
started them once. Stale NATS JetStream data from previous sessions
caused services to bind to wrong topics. Fix: stop and restart flows
on every config push that includes flow definitions.
2. Gateway publishToTopic sent messages without an id property. Pipeline
FlowProcessor handlers check properties.id and silently return if
missing. Fix: auto-generate a message id when publishing to topics.
Both fixes validated: 13/13 integration tests passing, PDF decoder
correctly receives and processes document messages through the pipeline.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wire up the query and retrieval side of the pipeline so the agent can
answer questions from stored knowledge:
- Triples query service (FalkorDB) — all SPO pattern queries via NATS
- Graph embeddings query service (Qdrant) — entity vector similarity
- Document embeddings query service (Qdrant) — chunk vector similarity
- Graph RAG service — full concept→entity→traverse→score→synthesize pipeline
- Document RAG service — embed→find chunks→synthesize pipeline
- Runner scripts for chunker, extractor, embeddings (missing from Phase 5)
- Add DocumentEmbeddingsRequest/Response schema types
- Add RAG prompt templates (extract-concepts, edge-scoring, synthesize)
- Add graph/doc embeddings query topics to seed config + flow manager
- Add all pipeline/query/retrieval services to docker-compose
- 8 new runner scripts, 8 new pnpm script aliases
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add end-to-end document processing pipeline:
- PDF decoder service (pdfjs-dist) extracts text per page from librarian docs
- Ollama native LLM service for local model inference
- FalkorDB triples store FlowProcessor consumer
- Qdrant graph embeddings store FlowProcessor consumer
- Fix spec name collisions in chunker/extractor (input→chunk-input, etc.)
- Gateway /load endpoint to trigger document processing
- Align flow manager blueprint and seed config with full pipeline topics
- Add runner scripts and test coverage for document load
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Flow Management Service:
- FlowManagerService (AsyncProcessor) handling list/get/start/stop flows
and list/get blueprints via kebab-case wire format
- Default blueprint with all service topic mappings
- Pushes flow config to config service on start/stop
Config Seeding:
- seed-config.ts script pushes prompt templates (extract-relationships,
extract-definitions, document-prompt, kg-prompt) and default flow
definition via gateway REST API
Integration Tests:
- Librarian CRUD: add-document, list-documents, get-content, delete
- Agent query: verifies routing through gateway to agent service
- Skip flags: SKIP_LIBRARIAN=1, SKIP_AGENT=1
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Minor fixes from linter: readonly modifiers, unused parameter prefixes,
type narrowing in graph-rag BFS traversal and edge scoring.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
