mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-07-10 05:42:12 +02:00
Adds a sparse keyword retrieval path beside the existing vector path in document-RAG, fused by weighted Reciprocal Rank Fusion on chunk_id, behind --retrieval-mode (vector | keyword | hybrid, default vector). The keyword index is a new pluggable service (KeywordIndexService / KeywordIndexClientSpec); the first backend is SQLite FTS5, consuming Chunk messages off the ingestion stream and answering BM25 queries from one process, since the index is a single local file. Query text is sanitized into per-term quoted phrases (raw text is not valid FTS5 syntax), which also makes dotted clause numbers and error codes exact-match without a trigram index. Indexes are scoped per (workspace, collection) and dropped on collection deletion. The keyword-index client spec is only registered when the sparse path is enabled, so existing flow definitions without keyword-index queues are untouched; with retrieval_mode=vector the retrieval path is unchanged. In hybrid mode a keyword-path failure degrades to vector-only.
This commit is contained in:
parent
e5206bddd0
commit
2bdc930b2a
16 changed files with 1013 additions and 13 deletions
|
|
@ -76,6 +76,7 @@ document-embeddings = "trustgraph.embeddings.document_embeddings:run"
|
|||
document-rag = "trustgraph.retrieval.document_rag:run"
|
||||
embeddings-fastembed = "trustgraph.embeddings.fastembed:run"
|
||||
embeddings-ollama = "trustgraph.embeddings.ollama:run"
|
||||
kw-index-fts5 = "trustgraph.storage.kw_index.fts5:run"
|
||||
graph-embeddings-query-milvus = "trustgraph.query.graph_embeddings.milvus:run"
|
||||
graph-embeddings-query-pinecone = "trustgraph.query.graph_embeddings.pinecone:run"
|
||||
graph-embeddings-query-qdrant = "trustgraph.query.graph_embeddings.qdrant:run"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue