mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-04-25 08:26:21 +02:00
286f762369
The id field in pipeline Metadata was being overwritten at each processing stage (document → page → chunk), causing knowledge storage to create separate cores per chunk instead of grouping by document. Add a root field that: - Is set by librarian to the original document ID - Is copied unchanged through PDF decoder, chunkers, and extractors - Is used by knowledge storage for document_id grouping (with fallback to id) Changes: - Add root field to Metadata schema with empty string default - Set root=document.id in librarian when initiating document processing - Copy root through PDF decoder, recursive chunker, and all extractors - Update knowledge storage to use root (or id as fallback) for grouping - Add root handling to translators and gateway serialization - Update test mock Metadata class to include root parameter |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| agent.py | ||
| base.py | ||
| collection.py | ||
| config.py | ||
| diagnosis.py | ||
| document_loading.py | ||
| embeddings.py | ||
| embeddings_query.py | ||
| flow.py | ||
| knowledge.py | ||
| library.py | ||
| metadata.py | ||
| nlp_query.py | ||
| primitives.py | ||
| prompt.py | ||
| retrieval.py | ||
| rows_query.py | ||
| structured_query.py | ||
| text_completion.py | ||
| tool.py | ||
| triples.py | ||