apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-07-27 22:21:04 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

feat: add Docling-based document decoder as alternative to unstructured (#1054)

Browse source 
Add trustgraph-docling package providing a document decoder powered by
IBM's Docling library. Supports PDF, DOCX, XLSX, PPTX, HTML, Markdown,
and CSV with two chunking modes:

- page mode (default): groups output by page for page-based formats,
  emits TextDocument messages for the existing downstream chunker
- hybrid mode: uses Docling's HybridChunker for structure-aware
  semantic chunking, emits Chunk messages directly

Tables are preserved as HTML markup. Images are stored in the librarian
with provenance but excluded from the text pipeline. Full provenance
chain is maintained for explainability tracing.

Includes Containerfile, CI/CD integration (PR checks and release
workflows), Makefile targets, and tech spec documentation.

Added unit tests for the package.
This commit is contained in:
cybermaggedon 2026-07-20 21:25:33 +01:00 • committed by GitHub
parent 6d51773e00
commit bdfdbb3c1f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 1733 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

6
Makefile
View file

@ -18,6 +18,7 @@ wheels:
pip3 wheel --no-deps --wheel-dir dist trustgraph-cli/
pip3 wheel --no-deps --wheel-dir dist trustgraph-ocr/
pip3 wheel --no-deps --wheel-dir dist trustgraph-unstructured/
pip3 wheel --no-deps --wheel-dir dist trustgraph-docling/
pip3 wheel --no-deps --wheel-dir dist trustgraph-mcp/
packages: update-package-versions
@ -31,6 +32,7 @@ packages: update-package-versions
cd trustgraph-cli && python -m build --sdist --outdir ../dist/
cd trustgraph-ocr && python -m build --sdist --outdir ../dist/
cd trustgraph-unstructured && python -m build --sdist --outdir ../dist/
cd trustgraph-docling && python -m build --sdist --outdir ../dist/
cd trustgraph-mcp && python -m build --sdist --outdir ../dist/
pypi-upload:
@ -49,13 +51,14 @@ update-package-versions:
echo __version__ = \"${VERSION}\" > trustgraph-cli/trustgraph/cli_version.py
echo __version__ = \"${VERSION}\" > trustgraph-ocr/trustgraph/ocr_version.py
echo __version__ = \"${VERSION}\" > trustgraph-unstructured/trustgraph/unstructured_version.py
echo __version__ = \"${VERSION}\" > trustgraph-docling/trustgraph/docling_version.py
echo __version__ = \"${VERSION}\" > trustgraph/trustgraph/trustgraph_version.py
echo __version__ = \"${VERSION}\" > trustgraph-mcp/trustgraph/mcp_version.py
containers: container-base container-flow \
container-bedrock container-vertexai \
container-hf container-ocr \
container-unstructured container-mcp
container-unstructured container-docling container-mcp
some-containers: container-base container-flow
# container-unstructured
@ -68,6 +71,7 @@ push:
${DOCKER} push ${CONTAINER_BASE}/trustgraph-hf:${VERSION}
${DOCKER} push ${CONTAINER_BASE}/trustgraph-ocr:${VERSION}
${DOCKER} push ${CONTAINER_BASE}/trustgraph-unstructured:${VERSION}
${DOCKER} push ${CONTAINER_BASE}/trustgraph-docling:${VERSION}
${DOCKER} push ${CONTAINER_BASE}/trustgraph-mcp:${VERSION}
# Individual container build targets