trustgraph/docs/tech-specs
cybermaggedon bdfdbb3c1f
feat: add Docling-based document decoder as alternative to unstructured (#1054)
Add trustgraph-docling package providing a document decoder powered by
IBM's Docling library. Supports PDF, DOCX, XLSX, PPTX, HTML, Markdown,
and CSV with two chunking modes:

- page mode (default): groups output by page for page-based formats,
  emits TextDocument messages for the existing downstream chunker
- hybrid mode: uses Docling's HybridChunker for structure-aware
  semantic chunking, emits Chunk messages directly

Tables are preserved as HTML markup. Images are stored in the librarian
with provenance but excluded from the text pipeline. Full provenance
chain is maintained for explainability tracing.

Includes Containerfile, CI/CD integration (PR checks and release
workflows), Makefile targets, and tech spec documentation.

Added unit tests for the package.
2026-07-20 21:25:33 +01:00
..
ar Structure the tech specs directory (#836) 2026-04-21 16:06:41 +01:00
es Structure the tech specs directory (#836) 2026-04-21 16:06:41 +01:00
he Structure the tech specs directory (#836) 2026-04-21 16:06:41 +01:00
hi Structure the tech specs directory (#836) 2026-04-21 16:06:41 +01:00
pt Structure the tech specs directory (#836) 2026-04-21 16:06:41 +01:00
ru Structure the tech specs directory (#836) 2026-04-21 16:06:41 +01:00
sw Structure the tech specs directory (#836) 2026-04-21 16:06:41 +01:00
tr Structure the tech specs directory (#836) 2026-04-21 16:06:41 +01:00
zh-cn Structure the tech specs directory (#836) 2026-04-21 16:06:41 +01:00
__TEMPLATE.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
active-flow-key-restructure.md Flow service lifecycle management (#822) 2026-04-16 17:19:39 +01:00
agent-explainability.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
agent-orchestration.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
architecture-principles.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
audit-events.md feat: add structured audit event system (#1027) 2026-07-06 10:47:49 +01:00
bootstrap.md feat: pluggable bootstrap framework with ordered initialisers (#847) 2026-04-22 18:03:46 +01:00
capabilities.md fix: list-my-workspaces permissions were broken (#1002) 2026-06-29 09:13:05 +01:00
cassandra-consolidation.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
cassandra-performance-refactor.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
collection-management.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
config-push-poke.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
data-ownership-model.md refactor(iam): pluggable IAM regime via authenticate/authorise contract (#853) 2026-04-28 16:19:41 +01:00
docling-decoder.md feat: add Docling-based document decoder as alternative to unstructured (#1054) 2026-07-20 21:25:33 +01:00
document-embeddings-chunk-id.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
embeddings-batch-processing.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
entity-centric-graph.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
explainability-cli.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
extraction-flows.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
extraction-provenance-subgraph.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
extraction-time-provenance.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
flow-blueprint-definition.md feat: pluggable image-to-text service with OpenAI vision backend (#1038) 2026-07-12 12:47:04 +01:00
flow-configurable-parameters.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
flow-service-queue-lifecycle.md Flow service lifecycle management (#822) 2026-04-16 17:19:39 +01:00
graph-contexts.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
graph-rag-semantic-filter.md feat: filter and cap GraphRAG reranker input across full stack (#1021) 2026-07-03 15:51:04 +01:00
graphql-query.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
graphrag-performance-optimization.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
iam-contract.md iam: self-service ops, optional workspace filters, Mux service routing (#855) 2026-04-28 22:13:12 +01:00
iam-protocol.md feat: add list-my-workspaces operation and document IAM in API specs (#961) 2026-05-29 19:17:37 +01:00
iam.md iam: self-service ops, optional workspace filters, Mux service routing (#855) 2026-04-28 22:13:12 +01:00
import-export-graceful-shutdown.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
jsonl-prompt-output.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
kafka-backend.md Add Kafka pub/sub backend (#830) 2026-04-18 11:18:34 +01:00
knowledge-core-completeness.md feat: complete knowledge core storage — named graphs, provenance, source material (#973) 2026-06-03 10:46:52 +01:00
large-document-loading.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
logging-strategy.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
mcp-tool-arguments.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
mcp-tool-bearer-token.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
minio-to-s3-migration.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
more-config-cli.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
multi-tenant-support.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
neo4j-user-collection-isolation.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
no-auth-regime.md feat: add no-auth IAM regime as a drop-in replacement for iam-svc (#933) 2026-05-18 14:10:05 +01:00
ontology-extract-phase-2.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
ontology.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
ontorag.md Fix ontology selector defaults, add bypass mode, enforce domain/range (#929) 2026-05-16 15:13:38 +01:00
openapi-spec.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
pubsub-abstraction.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
pubsub.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
python-api-refactor.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
query-time-explainability.md feat: structured source document references in graph-rag responses (#1035) 2026-07-08 23:59:56 +01:00
rag-streaming-support.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
schema-refactoring-proposal.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
sparql-query.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
streaming-llm-responses.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
structured-data-2.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
structured-data-descriptor.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
structured-data-schemas.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
structured-data.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
structured-diag-service.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
structured-output.md feat: LLM-native structured output via JSON schema enforcement (#1037) 2026-07-10 15:28:56 +01:00
tool-group.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
tool-services.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
universal-decoder.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
vector-store-lifecycle.md Feat: TrustGraph i18n & Documentation Translation Updates (#781) 2026-04-14 12:08:32 +01:00
workspace-scoped-services.md Per-workspace queue routing for workspace-scoped services (#862) 2026-05-04 10:30:03 +01:00