Updated the README to clarify the concept of holons and the functionality of TrustGraph. Improved the structure and flow of information regarding context management and agent explainability.
Replace the broken GATEWAY_SECRET auth (token was sent as a query
parameter, silently ignored by the gateway) with end-to-end Bearer
token forwarding. Each MCP caller gets a dedicated WebSocket
authenticated via the gateway's in-band first-frame protocol, with
whoami verification on first connect.
Also fix and extend the tool surface:
- embeddings: accept list of texts (was single string)
- triples_query: use Term wire format with compact keys (was legacy
Value format), add collection and graph parameters
- sparql_query: new tool for SPARQL SELECT/ASK/CONSTRUCT/DESCRIBE
- graphql_query: new tool for structured data (rows) GraphQL queries
- all tools: add optional workspace parameter
Replace the broken GATEWAY_SECRET auth (token was sent as a query
parameter, silently ignored by the gateway) with end-to-end Bearer
token forwarding. Each MCP caller gets a dedicated WebSocket
authenticated via the gateway's in-band first-frame protocol, with
whoami verification on first connect.
Also fix and extend the tool surface:
- embeddings: accept list of texts (was single string)
- triples_query: use Term wire format with compact keys (was legacy
Value format), add collection and graph parameters
- sparql_query: new tool for SPARQL SELECT/ASK/CONSTRUCT/DESCRIBE
- graphql_query: new tool for structured data (rows) GraphQL queries
- all tools: add optional workspace parameter
The query service now uses async_execute_paged (indexed path) and
async_scan (scan path) instead of async_execute. Tests were mocking
the old function, causing them to hang indefinitely.
- Pass auth token to schema discovery and descriptor generation in
tg-load-structured-data, fixing 401 errors with IAM enabled
- Fix row query pagination: replace single-page async_execute with
async_scan that streams pages and applies filters without
materialising the full result set (OOM on large datasets)
- Add missing filter operators (not, startsWith, endsWith, not_in)
to row query post-filter matching
- Fall back to scan path when an indexed field is queried with an
empty string value, since empty index values are not stored
- Revert top-level indexes array support — the current table schema
overwrites rows with duplicate index values, so only primary_key
fields are safe to index until the schema is redesigned
resolve_cassandra_config did not accept replication_factor as a kwarg,
so cassandra_replication_factor from YAML params was silently ignored
by all 6 callers. Add the kwarg and pass it from every caller.
Same fix for Qdrant: 3 writers now pass qdrant_replication_factor and
qdrant_shard_number from params.
Add tests covering the params path for both helpers.
- Add centralised qdrant_config.py helper with env-var fallback for
QDRANT_URL, QDRANT_API_KEY, QDRANT_REPLICATION_FACTOR, QDRANT_SHARD_NUMBER
- Update all 6 Qdrant processors to use the helper; writers pass
replication_factor and shard_number to create_collection
- Fix hardcoded Cassandra replication_factor=1 in cassandra_kg.py,
write.py, and sparql_cassandra.py to respect CASSANDRA_REPLICATION_FACTOR
- Upgrade Cassandra TLS from deprecated PROTOCOL_TLSv1_2 to
ssl.create_default_context() across all connectors
The librarian now reads OBJECT_STORE_ENDPOINT, OBJECT_STORE_ACCESS_KEY,
OBJECT_STORE_SECRET_KEY, OBJECT_STORE_REGION, and OBJECT_STORE_USE_SSL
from the environment when not set via params. This lets K8s Secrets
supply credentials without them appearing in launch.yaml.
Implements all three changes from the knowledge-core-completeness tech spec:
1. Named graph field preserved through Cassandra storage (7-element tuple),
enabling provenance triples to retain their graph URIs on round-trip.
2. Provenance triples already arrive on triples-input — no routing change
needed; Change 1 was sufficient.
3. Source material (library documents) streamed alongside triples and
embeddings during core download/upload. The knowledge manager fetches
the document hierarchy from the librarian on download and recreates it
on upload, preserving the full provenance chain across instances.
The mux unconditionally called auth.authorise() for every operation,
passing capability sentinels like AUTHENTICATED ("__authenticated__")
to the IAM regime. Since no role grants "__authenticated__", the regime
denied the request — breaking whoami (and any future AUTHENTICATED-only
operation) over the WebSocket path while the HTTP endpoints worked fine.
Match the guard pattern used by iam_endpoint.py and registry_endpoint.py:
only call authorise() for real capability strings, not sentinels.
Bulk clients (sync and async) were not forwarding the workspace parameter,
causing all bulk operations to hit the default workspace regardless of the
Api instance's workspace setting. Also fixes the gateway socket endpoint to
pass query parameters (including workspace) to the dispatcher, and prevents
the auth handshake from overwriting an explicitly set workspace.
Updates knowledge table store tests for paged query interface.
- Paginate heavy Cassandra reads (triples, graph/document embeddings)
using synchronous session.execute() in run_in_executor with fetch_size
paging, preventing materialization hang on large result sets
- Fix document stream endpoint to use workspace-scoped librarian queues
- Add decoder error handling for PDF/OCR/unstructured processors
- Add WebSocket mux guards for missing auth fields
- Add null check in librarian document streaming
- Rewrite get_document_content CLI to stream via librarian
- Add Poppler dependency to unstructured container
The tests were patching
trustgraph.embeddings.hf.hf.HuggingFaceEmbeddings - a module-level
attribute that doesn't exist because HuggingFaceEmbeddings is
imported locally inside _load_model. Changed all 8 occurrences to
patch langchain_huggingface.HuggingFaceEmbeddings, which is the
actual import source the code uses at runtime.
The auth-ok response includes the token's bound workspace, and
AsyncSocketClient was unconditionally adopting it — clobbering any
workspace the caller explicitly requested via the constructor.
Several CLI commands silently routed requests to the default workspace
regardless of the -w flag: show-flows, show-flow-blueprints,
show-parameter-types, set-prompt --system, and load-structured-data.
The workspace was sent in the inner request body but not on the
WebSocket envelope or API client constructor, so the gateway always
dispatched to the default workspace queue.
Add a new `list-my-workspaces` operation so non-admin users can
discover which workspaces they have access to. For OSS IAM, regular
users see their home workspace; admins see all workspaces.
Also add the full IAM service to both OpenAPI and AsyncAPI specs —
it was previously undocumented despite being a first-class service
on both HTTP and WebSocket interfaces.
Added click to the pip install line. Looks like huggingface-hub 1.13.0
added a CLI dependency on click but didn't declare it as a hard
requirement (or it's only required for the CLI entrypoint). Needed to
unblock the build.
Replaces the URL-based PDF downloads in tg-load-sample-documents with
seven curated, locally bundled documents covering diverse topics (recipes,
Belgian beer, trade routes, corporate scandals, pets, fortifications,
Bronze Age collapse). Documents are packaged as data files within
trustgraph-cli and loaded from metadata.json, removing the dependency
on external URLs and the doc-cache mechanism.
Replace removed `user` parameter with `workspace` support following
the tenancy axis change in #840. Adds -w/--workspace flag and
$TRUSTGRAPH_WORKSPACE env var.
Replace hallucinated relative imports with correct absolute imports
across the ontology query package, and fix OntologyMatcher reference
to match the actual class name OntologyMatcherForQueries. Simplify
test to use standard imports instead of importlib hack.
Cosmetic, but simpler imports provides undeterministic imports in a dev
environment, and also means we're properly testing linkage
