mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-07-07 12:22:11 +02:00
* feat: streaming N-Quads serializer for wire-format triples Groundwork for Phase 2 of #877 (knowledge export). Hand-rolled N-Triples term encoding: rdflib's term.n3() emits Turtle-style forms (numeric shorthand, unescaped newlines) that are invalid in line-oriented N-Quads, so literals are escaped per the ECHAR grammar and IRIs validated for representability. Round-trip tests parse the output back with rdflib's nquads parser and compare term-for-term. * feat: export/import workspace knowledge in .tgx bundles (#877) Phase 2 of the workspace bundle commands: tg-export-workspace now includes the workspace's knowledge by default — per-collection knowledge-graph triples as N-Quads (the collection names the graph, streamed through a tempfile so memory stays flat regardless of knowledge-base size) and the document library (metadata plus content, fetched one document at a time). --config-only skips knowledge on both sides; --triples-limit bounds very large graphs; -f/--flow-id selects the flow the triples services run through. tg-import-workspace streams triples back through the bulk import per collection and recreates library documents (children after parents). Knowledge import is additive, unlike config's skip-existing semantics. Embedding vectors are not carried in bundles: --process re-runs imported documents through the flow, which regenerates extraction output and embeddings; --process-collection targets it. Round-trip covered by unit tests over real archives: export with a mocked Api, re-import, and assert the bulk triples stream and library add calls reproduce the original values (including datatyped literals via the N-Quads path).
This commit is contained in:
parent
68e816e65c
commit
cfbd5b9079
6 changed files with 1040 additions and 181 deletions
|
|
@ -45,4 +45,18 @@ def sample_metadata():
|
|||
"metadata": [],
|
||||
"user": "test-user",
|
||||
"collection": "test-collection"
|
||||
}
|
||||
}
|
||||
|
||||
def iri(v):
|
||||
"""Wire-format IRI term dict, as triples_query_stream yields them."""
|
||||
return {"t": "i", "i": v}
|
||||
|
||||
|
||||
def lit(v, d=None, lang=None):
|
||||
"""Wire-format literal term dict (optional datatype / language)."""
|
||||
t = {"t": "l", "v": v}
|
||||
if d:
|
||||
t["d"] = d
|
||||
if lang:
|
||||
t["l"] = lang
|
||||
return t
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue