mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-07-27 06:01:01 +02:00
bdfdbb3c1f
Add trustgraph-docling package providing a document decoder powered by IBM's Docling library. Supports PDF, DOCX, XLSX, PPTX, HTML, Markdown, and CSV with two chunking modes: - page mode (default): groups output by page for page-based formats, emits TextDocument messages for the existing downstream chunker - hybrid mode: uses Docling's HybridChunker for structure-aware semantic chunking, emits Chunk messages directly Tables are preserved as HTML markup. Images are stored in the librarian with provenance but excluded from the text pipeline. Full provenance chain is maintained for explainability tracing. Includes Containerfile, CI/CD integration (PR checks and release workflows), Makefile targets, and tech spec documentation. Added unit tests for the package.
65 lines
1.4 KiB
YAML
65 lines
1.4 KiB
YAML
|
|
name: Test pull request
|
|
|
|
on:
|
|
pull_request:
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
|
|
test:
|
|
|
|
name: Run tests
|
|
runs-on: ubuntu-latest
|
|
|
|
container:
|
|
image: python:3.13
|
|
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v3
|
|
|
|
- name: Setup packages
|
|
run: make update-package-versions VERSION=2.7.999
|
|
|
|
- name: Setup environment
|
|
run: python3 -m venv env
|
|
|
|
- name: Invoke environment
|
|
run: . env/bin/activate
|
|
|
|
- name: Install trustgraph-base
|
|
run: (cd trustgraph-base; pip install .)
|
|
|
|
- name: Install trustgraph-cli
|
|
run: (cd trustgraph-cli; pip install .)
|
|
|
|
- name: Install trustgraph-flow
|
|
run: (cd trustgraph-flow; pip install .)
|
|
|
|
- name: Install trustgraph-unstructured
|
|
run: (cd trustgraph-unstructured; pip install .)
|
|
|
|
- name: Install trustgraph-docling
|
|
run: (cd trustgraph-docling; pip install .)
|
|
|
|
- name: Install trustgraph-vertexai
|
|
run: (cd trustgraph-vertexai; pip install .)
|
|
|
|
- name: Install trustgraph-bedrock
|
|
run: (cd trustgraph-bedrock; pip install .)
|
|
|
|
- name: Install some stuff
|
|
run: pip install pytest pytest-cov pytest-asyncio pytest-mock
|
|
|
|
- name: Unit tests
|
|
run: pytest tests/unit
|
|
|
|
- name: Integration tests (cut the out the long-running tests)
|
|
run: pytest tests/integration -m 'not slow'
|
|
|
|
- name: Contract tests
|
|
run: pytest tests/contract
|
|
|