trustgraph/trustgraph-flow/trustgraph/librarian
cybermaggedon 286f762369
The id field in pipeline Metadata was being overwritten at each processing (#686)
The id field in pipeline Metadata was being overwritten at each processing
stage (document → page → chunk), causing knowledge storage to create
separate cores per chunk instead of grouping by document.

Add a root field that:
- Is set by librarian to the original document ID
- Is copied unchanged through PDF decoder, chunkers, and extractors
- Is used by knowledge storage for document_id grouping (with fallback to id)

Changes:
- Add root field to Metadata schema with empty string default
- Set root=document.id in librarian when initiating document processing
- Copy root through PDF decoder, recursive chunker, and all extractors
- Update knowledge storage to use root (or id as fallback) for grouping
- Add root handling to translators and gateway serialization
- Update test mock Metadata class to include root parameter
2026-03-11 12:16:39 +00:00
..
__init__.py Librarian (#304) 2025-02-11 16:01:03 +00:00
__main__.py Librarian (#304) 2025-02-11 16:01:03 +00:00
blob_store.py Document API updates (#660) 2026-03-05 15:20:45 +00:00
collection_manager.py Fix collection existence test logic (#597) 2026-01-05 16:31:26 +00:00
librarian.py Filter answers from document lists (#683) 2026-03-10 17:22:19 +00:00
service.py The id field in pipeline Metadata was being overwritten at each processing (#686) 2026-03-11 12:16:39 +00:00