apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-07-09 13:22:10 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Merge branch 'master' into docs

Browse source
This commit is contained in:
Alex Jenkins 2026-04-10 23:10:52 -04:00 • committed by GitHub
commit ae58fa7f98
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
270 changed files with 19639 additions and 4087 deletions
Download patch file Download diff file Expand all files Collapse all files

108
docs/api-gateway-changes-v1.8-to-v2.1.md
View file

@ -1,108 +0,0 @@
# API Gateway Changes: v1.8 to v2.1
## Summary
The API gateway gained new WebSocket service dispatchers for embeddings
queries, a new REST streaming endpoint for document content, and underwent
a significant wire format change from `Value` to `Term`. The "objects"
service was renamed to "rows".
---
## New WebSocket Service Dispatchers
These are new request/response services available through the WebSocket
multiplexer at `/api/v1/socket` (flow-scoped):
| Service Key | Description |
|-------------|-------------|
| `document-embeddings` | Queries document chunks by text similarity. Request/response uses `DocumentEmbeddingsRequest`/`DocumentEmbeddingsResponse` schemas. |
| `row-embeddings` | Queries structured data rows by text similarity on indexed fields. Request/response uses `RowEmbeddingsRequest`/`RowEmbeddingsResponse` schemas. |
These join the existing `graph-embeddings` dispatcher (which was already
present in v1.8 but may have been updated).
### Full list of WebSocket flow service dispatchers (v2.1)
Request/response services (via `/api/v1/flow/{flow}/service/{kind}` or
WebSocket mux):
- `agent`, `text-completion`, `prompt`, `mcp-tool`
- `graph-rag`, `document-rag`
- `embeddings`, `graph-embeddings`, `document-embeddings`
- `triples`, `rows`, `nlp-query`, `structured-query`, `structured-diag`
- `row-embeddings`
---
## New REST Endpoint
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/document-stream` | Streams document content from the library as raw bytes. Query parameters: `user` (required), `document-id` (required), `chunk-size` (optional, default 1MB). Returns the document content in chunked transfer encoding, decoded from base64 internally. |
---
## Renamed Service: "objects" to "rows"
| v1.8 | v2.1 | Notes |
|------|------|-------|
| `objects_query.py` / `ObjectsQueryRequestor` | `rows_query.py` / `RowsQueryRequestor` | Schema changed from `ObjectsQueryRequest`/`ObjectsQueryResponse` to `RowsQueryRequest`/`RowsQueryResponse`. |
| `objects_import.py` / `ObjectsImport` | `rows_import.py` / `RowsImport` | Import dispatcher for structured data. |
The WebSocket service key changed from `"objects"` to `"rows"`, and the
import dispatcher key similarly changed from `"objects"` to `"rows"`.
---
## Wire Format Change: Value to Term
The serialization layer (`serialize.py`) was rewritten to use the new `Term`
type instead of the old `Value` type.
### Old format (v1.8 — `Value`)
```json
{"v": "http://example.org/entity", "e": true}
```
- `v`: the value (string)
- `e`: boolean flag indicating whether the value is a URI
### New format (v2.1 — `Term`)
IRIs:
```json
{"t": "i", "i": "http://example.org/entity"}
```
Literals:
```json
{"t": "l", "v": "some text", "d": "datatype-uri", "l": "en"}
```
Quoted triples (RDF-star):
```json
{"t": "r", "r": {"s": {...}, "p": {...}, "o": {...}}}
```
- `t`: type discriminator — `"i"` (IRI), `"l"` (literal), `"r"` (quoted triple), `"b"` (blank node)
- Serialization now delegates to `TermTranslator` and `TripleTranslator` from `trustgraph.messaging.translators.primitives`
### Other serialization changes
| Field | v1.8 | v2.1 |
|-------|------|------|
| Metadata | `metadata.metadata` (subgraph) | `metadata.root` (simple value) |
| Graph embeddings entity | `entity.vectors` (plural) | `entity.vector` (singular) |
| Document embeddings chunk | `chunk.vectors` + `chunk.chunk` (text) | `chunk.vector` + `chunk.chunk_id` (ID reference) |
---
## Breaking Changes
- **`Value` to `Term` wire format**: All clients sending/receiving triples, embeddings, or entity contexts through the gateway must update to the new Term format.
- **`objects` to `rows` rename**: WebSocket service key and import key changed.
- **Metadata field change**: `metadata.metadata` (a serialized subgraph) replaced by `metadata.root` (a simple value).
- **Embeddings field changes**: `vectors` (plural) became `vector` (singular); document embeddings now reference `chunk_id` instead of inline `chunk` text.
- **New `/api/v1/document-stream` endpoint**: Additive, not breaking.

176
docs/api.html
View file

File diff suppressed because one or more lines are too long

112
docs/cli-changes-v1.8-to-v2.1.md
View file

@ -1,112 +0,0 @@
# CLI Changes: v1.8 to v2.1
## Summary
The CLI (`trustgraph-cli`) has significant additions focused on three themes:
**explainability/provenance**, **embeddings access**, and **graph querying**.
Two legacy tools were removed, one was renamed, and several existing tools
gained new capabilities.
---
## New CLI Tools
### Explainability & Provenance
| Command | Description |
|---------|-------------|
| `tg-list-explain-traces` | Lists all explainability sessions (GraphRAG and Agent) in a collection, showing session IDs, type, question text, and timestamps. |
| `tg-show-explain-trace` | Displays the full explainability trace for a session. For GraphRAG: Question, Exploration, Focus, Synthesis stages. For Agent: Session, Iterations (thought/action/observation), Final Answer. Auto-detects trace type. Supports `--show-provenance` to trace edges back to source documents. |
| `tg-show-extraction-provenance` | Given a document ID, traverses the provenance chain: Document -> Pages -> Chunks -> Edges, using `prov:wasDerivedFrom` relationships. Supports `--show-content` and `--max-content` options. |
### Embeddings
| Command | Description |
|---------|-------------|
| `tg-invoke-embeddings` | Converts text to a vector embedding via the embeddings service. Accepts one or more text inputs, returns vectors as lists of floats. |
| `tg-invoke-graph-embeddings` | Queries graph entities by text similarity using vector embeddings. Returns matching entities with similarity scores. |
| `tg-invoke-document-embeddings` | Queries document chunks by text similarity using vector embeddings. Returns matching chunk IDs with similarity scores. |
| `tg-invoke-row-embeddings` | Queries structured data rows by text similarity on indexed fields. Returns matching rows with index values and scores. Requires `--schema-name` and supports `--index-name`. |
### Graph Querying
| Command | Description |
|---------|-------------|
| `tg-query-graph` | Pattern-based triple store query. Unlike `tg-show-graph` (which dumps everything), this allows selective queries by any combination of subject, predicate, object, and graph. Auto-detects value types: IRIs (`http://...`, `urn:...`, `<...>`), quoted triples (`<<s p o>>`), and literals. |
| `tg-get-document-content` | Retrieves document content from the library by document ID. Can output to file or stdout, handles both text and binary content. |
---
## Removed CLI Tools
| Command | Notes |
|---------|-------|
| `tg-load-pdf` | Removed. Document loading is now handled through the library/processing pipeline. |
| `tg-load-text` | Removed. Document loading is now handled through the library/processing pipeline. |
---
## Renamed CLI Tools
| Old Name | New Name | Notes |
|----------|----------|-------|
| `tg-invoke-objects-query` | `tg-invoke-rows-query` | Reflects the terminology rename from "objects" to "rows" for structured data. |
---
## Significant Changes to Existing Tools
### `tg-invoke-graph-rag`
- **Explainability support**: Now supports a 4-stage explainability pipeline (Question, Grounding/Exploration, Focus, Synthesis) with inline provenance event display.
- **Streaming**: Uses WebSocket streaming for real-time output.
- **Provenance tracing**: Can trace selected edges back to source documents via reification and `prov:wasDerivedFrom` chains.
- Grew from ~30 lines to ~760 lines to accommodate the full explainability pipeline.
### `tg-invoke-document-rag`
- **Explainability support**: Added `question_explainable()` mode that streams Document RAG responses with inline provenance events (Question, Grounding, Exploration, Synthesis stages).
### `tg-invoke-agent`
- **Explainability support**: Added `question_explainable()` mode showing provenance events inline during agent execution (Question, Analysis, Conclusion, AgentThought, AgentObservation, AgentAnswer).
- Verbose mode shows thought/observation streams with emoji prefixes.
### `tg-show-graph`
- **Streaming mode**: Now uses `triples_query_stream()` with configurable batch sizes for lower time-to-first-result and reduced memory overhead.
- **Named graph support**: New `--graph` filter option. Recognises named graphs:
- Default graph (empty): Core knowledge facts
- `urn:graph:source`: Extraction provenance
- `urn:graph:retrieval`: Query-time explainability
- **Show graph column**: New `--show-graph` flag to display the named graph for each triple.
- **Configurable limits**: New `--limit` and `--batch-size` options.
### `tg-graph-to-turtle`
- **RDF-star support**: Now handles quoted triples (RDF-star reification).
- **Streaming mode**: Uses streaming for lower time-to-first-processing.
- **Wire format handling**: Updated to use the new term wire format (`{"t": "i", "i": uri}` for IRIs, `{"t": "l", "v": value}` for literals, `{"t": "r", "r": {...}}` for quoted triples).
- **Named graph support**: New `--graph` filter option.
### `tg-set-tool`
- **New tool type**: `row-embeddings-query` for semantic search on structured data indexes.
- **New options**: `--schema-name`, `--index-name`, `--limit` for configuring row embeddings query tools.
### `tg-show-tools`
- Displays the new `row-embeddings-query` tool type with its `schema-name`, `index-name`, and `limit` fields.
### `tg-load-knowledge`
- **Progress reporting**: Now counts and reports triples and entity contexts loaded per file and in total.
- **Term format update**: Entity contexts now use the new Term format (`{"t": "i", "i": uri}`) instead of the old Value format (`{"v": entity, "e": True}`).
---
## Breaking Changes
- **Terminology rename**: The `Value` schema was renamed to `Term` across the system (PR #622). This affects the wire format used by CLI tools that interact with the graph store. The new format uses `{"t": "i", "i": uri}` for IRIs and `{"t": "l", "v": value}` for literals, replacing the old `{"v": ..., "e": ...}` format.
- **`tg-invoke-objects-query` renamed** to `tg-invoke-rows-query`.
- **`tg-load-pdf` and `tg-load-text` removed**.

683
docs/python-api.md
View file

@ -911,7 +911,7 @@ results = flow.graph_embeddings_query(
# results contains {"entities": [{"entity": {...}, "score": 0.95}, ...]}
```
### `graph_rag(self, query, user='trustgraph', collection='default', entity_limit=50, triple_limit=30, max_subgraph_size=150, max_path_length=2)`
### `graph_rag(self, query, user='trustgraph', collection='default', entity_limit=50, triple_limit=30, max_subgraph_size=150, max_path_length=2, edge_score_limit=30, edge_limit=25)`
Execute graph-based Retrieval-Augmented Generation (RAG) query.
@ -927,6 +927,8 @@ traversing entity relationships, then generates a response using an LLM.
- `triple_limit`: Maximum triples per entity (default: 30)
- `max_subgraph_size`: Maximum total triples in subgraph (default: 150)
- `max_path_length`: Maximum traversal depth (default: 2)
- `edge_score_limit`: Max edges for semantic pre-filter (default: 50)
- `edge_limit`: Max edges after LLM scoring (default: 25)
**Returns:** str: Generated response incorporating graph context
@ -1216,6 +1218,23 @@ Select matching schemas for a data sample using prompt analysis.
**Returns:** dict with schema_matches array and metadata
### `sparql_query(self, query, user='trustgraph', collection='default', limit=10000)`
Execute a SPARQL query against the knowledge graph.
**Arguments:**
- `query`: SPARQL 1.1 query string
- `user`: User/keyspace identifier (default: "trustgraph")
- `collection`: Collection identifier (default: "default")
- `limit`: Safety limit on results (default: 10000)
**Returns:** dict with query results. Structure depends on query type: - SELECT: {"query-type": "select", "variables": [...], "bindings": [...]} - ASK: {"query-type": "ask", "ask-result": bool} - CONSTRUCT/DESCRIBE: {"query-type": "construct", "triples": [...]}
**Raises:**
- `ProtocolException`: If an error occurs
### `structured_query(self, question, user='trustgraph', collection='default')`
Execute a natural language question against structured data.
@ -1937,54 +1956,24 @@ for triple in results.get("triples", []):
from trustgraph.api import SocketClient
```
Synchronous WebSocket client for streaming operations.
Synchronous WebSocket client with persistent connection.
Provides a synchronous interface to WebSocket-based TrustGraph services,
wrapping async websockets library with synchronous generators for ease of use.
Supports streaming responses from agents, RAG queries, and text completions.
Note: This is a synchronous wrapper around async WebSocket operations. For
true async support, use AsyncSocketClient instead.
Maintains a single websocket connection and multiplexes requests
by ID via a background reader task. Provides synchronous generators
for streaming responses.
### Methods
### `__init__(self, url: str, timeout: int, token: str | None) -> None`
Initialize synchronous WebSocket client.
**Arguments:**
- `url`: Base URL for TrustGraph API (HTTP/HTTPS will be converted to WS/WSS)
- `timeout`: WebSocket timeout in seconds
- `token`: Optional bearer token for authentication
Initialize self. See help(type(self)) for accurate signature.
### `close(self) -> None`
Close WebSocket connections.
Note: Cleanup is handled automatically by context managers in async code.
Close the persistent WebSocket connection.
### `flow(self, flow_id: str) -> 'SocketFlowInstance'`
Get a flow instance for WebSocket streaming operations.
**Arguments:**
- `flow_id`: Flow identifier
**Returns:** SocketFlowInstance: Flow instance with streaming methods
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
# Stream agent responses
for chunk in flow.agent(question="Hello", user="trustgraph", streaming=True):
print(chunk.content, end='', flush=True)
```
---
@ -1997,618 +1986,82 @@ from trustgraph.api import SocketFlowInstance
Synchronous WebSocket flow instance for streaming operations.
Provides the same interface as REST FlowInstance but with WebSocket-based
streaming support for real-time responses. All methods support an optional
`streaming` parameter to enable incremental result delivery.
streaming support for real-time responses.
### Methods
### `__init__(self, client: trustgraph.api.socket_client.SocketClient, flow_id: str) -> None`
Initialize socket flow instance.
**Arguments:**
- `client`: Parent SocketClient
- `flow_id`: Flow identifier
Initialize self. See help(type(self)) for accurate signature.
### `agent(self, question: str, user: str, state: Dict[str, Any] | None = None, group: str | None = None, history: List[Dict[str, Any]] | None = None, streaming: bool = False, **kwargs: Any) -> Dict[str, Any] | Iterator[trustgraph.api.types.StreamingChunk]`
Execute an agent operation with streaming support.
Agents can perform multi-step reasoning with tool use. This method always
returns streaming chunks (thoughts, observations, answers) even when
streaming=False, to show the agent's reasoning process.
**Arguments:**
- `question`: User question or instruction
- `user`: User identifier
- `state`: Optional state dictionary for stateful conversations
- `group`: Optional group identifier for multi-user contexts
- `history`: Optional conversation history as list of message dicts
- `streaming`: Enable streaming mode (default: False)
- `**kwargs`: Additional parameters passed to the agent service
**Returns:** Iterator[StreamingChunk]: Stream of agent thoughts, observations, and answers
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
# Stream agent reasoning
for chunk in flow.agent(
question="What is quantum computing?",
user="trustgraph",
streaming=True
):
if isinstance(chunk, AgentThought):
print(f"[Thinking] {chunk.content}")
elif isinstance(chunk, AgentObservation):
print(f"[Observation] {chunk.content}")
elif isinstance(chunk, AgentAnswer):
print(f"[Answer] {chunk.content}")
```
### `agent_explain(self, question: str, user: str, collection: str, state: Dict[str, Any] | None = None, group: str | None = None, history: List[Dict[str, Any]] | None = None, **kwargs: Any) -> Iterator[trustgraph.api.types.StreamingChunk | trustgraph.api.types.ProvenanceEvent]`
Execute an agent operation with explainability support.
Streams both content chunks (AgentThought, AgentObservation, AgentAnswer)
and provenance events (ProvenanceEvent). Provenance events contain URIs
that can be fetched using ExplainabilityClient to get detailed information
about the agent's reasoning process.
Agent trace consists of:
- Session: The initial question and session metadata
- Iterations: Each thought/action/observation cycle
- Conclusion: The final answer
**Arguments:**
- `question`: User question or instruction
- `user`: User identifier
- `collection`: Collection identifier for provenance storage
- `state`: Optional state dictionary for stateful conversations
- `group`: Optional group identifier for multi-user contexts
- `history`: Optional conversation history as list of message dicts
- `**kwargs`: Additional parameters passed to the agent service
- `Yields`:
- `Union[StreamingChunk, ProvenanceEvent]`: Agent chunks and provenance events
**Example:**
```python
from trustgraph.api import Api, ExplainabilityClient, ProvenanceEvent
from trustgraph.api import AgentThought, AgentObservation, AgentAnswer
socket = api.socket()
flow = socket.flow("default")
explain_client = ExplainabilityClient(flow)
provenance_ids = []
for item in flow.agent_explain(
question="What is the capital of France?",
user="trustgraph",
collection="default"
):
if isinstance(item, AgentThought):
print(f"[Thought] {item.content}")
elif isinstance(item, AgentObservation):
print(f"[Observation] {item.content}")
elif isinstance(item, AgentAnswer):
print(f"[Answer] {item.content}")
elif isinstance(item, ProvenanceEvent):
provenance_ids.append(item.explain_id)
# Fetch session trace after completion
if provenance_ids:
trace = explain_client.fetch_agent_trace(
provenance_ids[0], # Session URI is first
graph="urn:graph:retrieval",
user="trustgraph",
collection="default"
)
```
### `document_embeddings_query(self, text: str, user: str, collection: str, limit: int = 10, **kwargs: Any) -> Dict[str, Any]`
Query document chunks using semantic similarity.
**Arguments:**
- `text`: Query text for semantic search
- `user`: User/keyspace identifier
- `collection`: Collection identifier
- `limit`: Maximum number of results (default: 10)
- `**kwargs`: Additional parameters passed to the service
**Returns:** dict: Query results with chunk_ids of matching document chunks
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
results = flow.document_embeddings_query(
text="machine learning algorithms",
user="trustgraph",
collection="research-papers",
limit=5
)
# results contains {"chunks": [{"chunk_id": "...", "score": 0.95}, ...]}
```
### `document_rag(self, query: str, user: str, collection: str, doc_limit: int = 10, streaming: bool = False, **kwargs: Any) -> str | Iterator[str]`
Execute document-based RAG query with optional streaming.
Uses vector embeddings to find relevant document chunks, then generates
a response using an LLM. Streaming mode delivers results incrementally.
**Arguments:**
- `query`: Natural language query
- `user`: User/keyspace identifier
- `collection`: Collection identifier
- `doc_limit`: Maximum document chunks to retrieve (default: 10)
- `streaming`: Enable streaming mode (default: False)
- `**kwargs`: Additional parameters passed to the service
**Returns:** Union[str, Iterator[str]]: Complete response or stream of text chunks
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
# Streaming document RAG
for chunk in flow.document_rag(
query="Summarize the key findings",
user="trustgraph",
collection="research-papers",
doc_limit=5,
streaming=True
):
print(chunk, end='', flush=True)
```
### `document_rag_explain(self, query: str, user: str, collection: str, doc_limit: int = 10, **kwargs: Any) -> Iterator[trustgraph.api.types.RAGChunk | trustgraph.api.types.ProvenanceEvent]`
Execute document-based RAG query with explainability support.
Streams both content chunks (RAGChunk) and provenance events (ProvenanceEvent).
Provenance events contain URIs that can be fetched using ExplainabilityClient
to get detailed information about how the response was generated.
Document RAG trace consists of:
- Question: The user's query
- Exploration: Chunks retrieved from document store (chunk_count)
- Synthesis: The generated answer
**Arguments:**
- `query`: Natural language query
- `user`: User/keyspace identifier
- `collection`: Collection identifier
- `doc_limit`: Maximum document chunks to retrieve (default: 10)
- `**kwargs`: Additional parameters passed to the service
- `Yields`:
- `Union[RAGChunk, ProvenanceEvent]`: Content chunks and provenance events
**Example:**
```python
from trustgraph.api import Api, ExplainabilityClient, RAGChunk, ProvenanceEvent
socket = api.socket()
flow = socket.flow("default")
explain_client = ExplainabilityClient(flow)
for item in flow.document_rag_explain(
query="Summarize the key findings",
user="trustgraph",
collection="research-papers",
doc_limit=5
):
if isinstance(item, RAGChunk):
print(item.content, end='', flush=True)
elif isinstance(item, ProvenanceEvent):
# Fetch entity details
entity = explain_client.fetch_entity(
item.explain_id,
graph=item.explain_graph,
user="trustgraph",
collection="research-papers"
)
print(f"Event: {entity}", file=sys.stderr)
```
### `embeddings(self, texts: list, **kwargs: Any) -> Dict[str, Any]`
Generate vector embeddings for one or more texts.
**Arguments:**
- `texts`: List of input texts to embed
- `**kwargs`: Additional parameters passed to the service
**Returns:** dict: Response containing vectors (one set per input text)
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
result = flow.embeddings(["quantum computing"])
vectors = result.get("vectors", [])
```
### `graph_embeddings_query(self, text: str, user: str, collection: str, limit: int = 10, **kwargs: Any) -> Dict[str, Any]`
Query knowledge graph entities using semantic similarity.
**Arguments:**
- `text`: Query text for semantic search
- `user`: User/keyspace identifier
- `collection`: Collection identifier
- `limit`: Maximum number of results (default: 10)
- `**kwargs`: Additional parameters passed to the service
**Returns:** dict: Query results with similar entities
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
results = flow.graph_embeddings_query(
text="physicist who discovered radioactivity",
user="trustgraph",
collection="scientists",
limit=5
)
```
### `graph_rag(self, query: str, user: str, collection: str, max_subgraph_size: int = 1000, max_subgraph_count: int = 5, max_entity_distance: int = 3, streaming: bool = False, **kwargs: Any) -> str | Iterator[str]`
### `graph_rag(self, query: str, user: str, collection: str, entity_limit: int = 50, triple_limit: int = 30, max_subgraph_size: int = 1000, max_path_length: int = 2, edge_score_limit: int = 30, edge_limit: int = 25, streaming: bool = False, **kwargs: Any) -> str | Iterator[str]`
Execute graph-based RAG query with optional streaming.
Uses knowledge graph structure to find relevant context, then generates
a response using an LLM. Streaming mode delivers results incrementally.
**Arguments:**
- `query`: Natural language query
- `user`: User/keyspace identifier
- `collection`: Collection identifier
- `max_subgraph_size`: Maximum total triples in subgraph (default: 1000)
- `max_subgraph_count`: Maximum number of subgraphs (default: 5)
- `max_entity_distance`: Maximum traversal depth (default: 3)
- `streaming`: Enable streaming mode (default: False)
- `**kwargs`: Additional parameters passed to the service
**Returns:** Union[str, Iterator[str]]: Complete response or stream of text chunks
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
# Streaming graph RAG
for chunk in flow.graph_rag(
query="Tell me about Marie Curie",
user="trustgraph",
collection="scientists",
streaming=True
):
print(chunk, end='', flush=True)
```
### `graph_rag_explain(self, query: str, user: str, collection: str, max_subgraph_size: int = 1000, max_subgraph_count: int = 5, max_entity_distance: int = 3, **kwargs: Any) -> Iterator[trustgraph.api.types.RAGChunk | trustgraph.api.types.ProvenanceEvent]`
### `graph_rag_explain(self, query: str, user: str, collection: str, entity_limit: int = 50, triple_limit: int = 30, max_subgraph_size: int = 1000, max_path_length: int = 2, edge_score_limit: int = 30, edge_limit: int = 25, **kwargs: Any) -> Iterator[trustgraph.api.types.RAGChunk | trustgraph.api.types.ProvenanceEvent]`
Execute graph-based RAG query with explainability support.
Streams both content chunks (RAGChunk) and provenance events (ProvenanceEvent).
Provenance events contain URIs that can be fetched using ExplainabilityClient
to get detailed information about how the response was generated.
**Arguments:**
- `query`: Natural language query
- `user`: User/keyspace identifier
- `collection`: Collection identifier
- `max_subgraph_size`: Maximum total triples in subgraph (default: 1000)
- `max_subgraph_count`: Maximum number of subgraphs (default: 5)
- `max_entity_distance`: Maximum traversal depth (default: 3)
- `**kwargs`: Additional parameters passed to the service
- `Yields`:
- `Union[RAGChunk, ProvenanceEvent]`: Content chunks and provenance events
**Example:**
```python
from trustgraph.api import Api, ExplainabilityClient, RAGChunk, ProvenanceEvent
socket = api.socket()
flow = socket.flow("default")
explain_client = ExplainabilityClient(flow)
provenance_ids = []
response_text = ""
for item in flow.graph_rag_explain(
query="Tell me about Marie Curie",
user="trustgraph",
collection="scientists"
):
if isinstance(item, RAGChunk):
response_text += item.content
print(item.content, end='', flush=True)
elif isinstance(item, ProvenanceEvent):
provenance_ids.append(item.provenance_id)
# Fetch explainability details
for prov_id in provenance_ids:
entity = explain_client.fetch_entity(
prov_id,
graph="urn:graph:retrieval",
user="trustgraph",
collection="scientists"
)
print(f"Entity: {entity}")
```
### `mcp_tool(self, name: str, parameters: Dict[str, Any], **kwargs: Any) -> Dict[str, Any]`
Execute a Model Context Protocol (MCP) tool.
**Arguments:**
- `name`: Tool name/identifier
- `parameters`: Tool parameters dictionary
- `**kwargs`: Additional parameters passed to the service
**Returns:** dict: Tool execution result
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
result = flow.mcp_tool(
name="search-web",
parameters={"query": "latest AI news", "limit": 5}
)
```
### `prompt(self, id: str, variables: Dict[str, str], streaming: bool = False, **kwargs: Any) -> str | Iterator[str]`
Execute a prompt template with optional streaming.
**Arguments:**
- `id`: Prompt template identifier
- `variables`: Dictionary of variable name to value mappings
- `streaming`: Enable streaming mode (default: False)
- `**kwargs`: Additional parameters passed to the service
**Returns:** Union[str, Iterator[str]]: Complete response or stream of text chunks
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
# Streaming prompt execution
for chunk in flow.prompt(
id="summarize-template",
variables={"topic": "quantum computing", "length": "brief"},
streaming=True
):
print(chunk, end='', flush=True)
```
### `row_embeddings_query(self, text: str, schema_name: str, user: str = 'trustgraph', collection: str = 'default', index_name: str | None = None, limit: int = 10, **kwargs: Any) -> Dict[str, Any]`
Query row data using semantic similarity on indexed fields.
Finds rows whose indexed field values are semantically similar to the
input text, using vector embeddings. This enables fuzzy/semantic matching
on structured data.
**Arguments:**
- `text`: Query text for semantic search
- `schema_name`: Schema name to search within
- `user`: User/keyspace identifier (default: "trustgraph")
- `collection`: Collection identifier (default: "default")
- `index_name`: Optional index name to filter search to specific index
- `limit`: Maximum number of results (default: 10)
- `**kwargs`: Additional parameters passed to the service
**Returns:** dict: Query results with matches containing index_name, index_value, text, and score
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
# Search for customers by name similarity
results = flow.row_embeddings_query(
text="John Smith",
schema_name="customers",
user="trustgraph",
collection="sales",
limit=5
)
# Filter to specific index
results = flow.row_embeddings_query(
text="machine learning engineer",
schema_name="employees",
index_name="job_title",
limit=10
)
```
### `rows_query(self, query: str, user: str, collection: str, variables: Dict[str, Any] | None = None, operation_name: str | None = None, **kwargs: Any) -> Dict[str, Any]`
Execute a GraphQL query against structured rows.
**Arguments:**
### `sparql_query_stream(self, query: str, user: str = 'trustgraph', collection: str = 'default', limit: int = 10000, batch_size: int = 20, **kwargs: Any) -> Iterator[Dict[str, Any]]`
- `query`: GraphQL query string
- `user`: User/keyspace identifier
- `collection`: Collection identifier
- `variables`: Optional query variables dictionary
- `operation_name`: Optional operation name for multi-operation documents
- `**kwargs`: Additional parameters passed to the service
**Returns:** dict: GraphQL response with data, errors, and/or extensions
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
query = '''
{
scientists(limit: 10) {
name
field
discoveries
}
}
'''
result = flow.rows_query(
query=query,
user="trustgraph",
collection="scientists"
)
```
Execute a SPARQL query with streaming batches.
### `text_completion(self, system: str, prompt: str, streaming: bool = False, **kwargs) -> str | Iterator[str]`
Execute text completion with optional streaming.
**Arguments:**
- `system`: System prompt defining the assistant's behavior
- `prompt`: User prompt/question
- `streaming`: Enable streaming mode (default: False)
- `**kwargs`: Additional parameters passed to the service
**Returns:** Union[str, Iterator[str]]: Complete response or stream of text chunks
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
# Non-streaming
response = flow.text_completion(
system="You are helpful",
prompt="Explain quantum computing",
streaming=False
)
print(response)
# Streaming
for chunk in flow.text_completion(
system="You are helpful",
prompt="Explain quantum computing",
streaming=True
):
print(chunk, end='', flush=True)
```
### `triples_query(self, s: str | Dict[str, Any] | None = None, p: str | Dict[str, Any] | None = None, o: str | Dict[str, Any] | None = None, g: str | None = None, user: str | None = None, collection: str | None = None, limit: int = 100, **kwargs: Any) -> List[Dict[str, Any]]`
Query knowledge graph triples using pattern matching.
**Arguments:**
- `s`: Subject filter - URI string, Term dict, or None for wildcard
- `p`: Predicate filter - URI string, Term dict, or None for wildcard
- `o`: Object filter - URI/literal string, Term dict, or None for wildcard
- `g`: Named graph filter - URI string or None for all graphs
- `user`: User/keyspace identifier (optional)
- `collection`: Collection identifier (optional)
- `limit`: Maximum results to return (default: 100)
- `**kwargs`: Additional parameters passed to the service
**Returns:** List[Dict]: List of matching triples in wire format
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
# Find all triples about a specific subject
triples = flow.triples_query(
s="http://example.org/person/marie-curie",
user="trustgraph",
collection="scientists"
)
# Query with named graph filter
triples = flow.triples_query(
s="urn:trustgraph:session:abc123",
g="urn:graph:retrieval",
user="trustgraph",
collection="default"
)
```
### `triples_query_stream(self, s: str | Dict[str, Any] | None = None, p: str | Dict[str, Any] | None = None, o: str | Dict[str, Any] | None = None, g: str | None = None, user: str | None = None, collection: str | None = None, limit: int = 100, batch_size: int = 20, **kwargs: Any) -> Iterator[List[Dict[str, Any]]]`
Query knowledge graph triples with streaming batches.
Yields batches of triples as they arrive, reducing time-to-first-result
and memory overhead for large result sets.
**Arguments:**
- `s`: Subject filter - URI string, Term dict, or None for wildcard
- `p`: Predicate filter - URI string, Term dict, or None for wildcard
- `o`: Object filter - URI/literal string, Term dict, or None for wildcard
- `g`: Named graph filter - URI string or None for all graphs
- `user`: User/keyspace identifier (optional)
- `collection`: Collection identifier (optional)
- `limit`: Maximum results to return (default: 100)
- `batch_size`: Triples per batch (default: 20)
- `**kwargs`: Additional parameters passed to the service
- `Yields`:
- `List[Dict]`: Batches of triples in wire format
**Example:**
```python
socket = api.socket()
flow = socket.flow("default")
for batch in flow.triples_query_stream(
user="trustgraph",
collection="default"
):
for triple in batch:
print(triple["s"], triple["p"], triple["o"])
```
---
@ -2618,17 +2071,35 @@ for batch in flow.triples_query_stream(
from trustgraph.api import AsyncSocketClient
```
Asynchronous WebSocket client
Asynchronous WebSocket client with persistent connection.
Maintains a single websocket connection and multiplexes requests
by ID, routing responses via a background reader task.
Use as an async context manager for proper lifecycle management:
async with AsyncSocketClient(url, timeout, token) as client:
result = await client._send_request(...)
Or call connect()/aclose() manually.
### Methods
### `__aenter__(self)`
### `__aexit__(self, exc_type, exc_val, exc_tb)`
### `__init__(self, url: str, timeout: int, token: str | None)`
Initialize self. See help(type(self)) for accurate signature.
### `aclose(self)`
Close WebSocket connection
Close the persistent WebSocket connection cleanly.
### `connect(self)`
Establish the persistent websocket connection.
### `flow(self, flow_id: str)`
@ -3151,7 +2622,10 @@ Detect whether a session is GraphRAG or Agent type.
Fetch the complete Agent trace starting from a session URI.
Follows the provenance chain: Question -> Analysis(s) -> Conclusion
Follows the provenance chain for all patterns:
- ReAct: Question -> Analysis(s) -> Conclusion
- Supervisor: Question -> Decomposition -> Finding(s) -> Synthesis
- Plan-then-Execute: Question -> Plan -> StepResult(s) -> Synthesis
**Arguments:**
@ -3162,7 +2636,7 @@ Follows the provenance chain: Question -> Analysis(s) -> Conclusion
- `api`: TrustGraph Api instance for librarian access (optional)
- `max_content`: Maximum content length for conclusion
**Returns:** Dict with question, iterations (Analysis list), conclusion entities
**Returns:** Dict with question, steps (mixed entity list), conclusion/synthesis
### `fetch_docrag_trace(self, question_uri: str, graph: str | None = None, user: str | None = None, collection: str | None = None, api: Any = None, max_content: int = 10000) -> Dict[str, Any]`
@ -3423,7 +2897,7 @@ Initialize self. See help(type(self)) for accurate signature.
from trustgraph.api import Analysis
```
Analysis entity - one think/act/observe cycle (Agent only).
Analysis+ToolUse entity - decision + tool call (Agent only).
**Fields:**
@ -3432,11 +2906,33 @@ Analysis entity - one think/act/observe cycle (Agent only).
- `action`: <class 'str'>
- `arguments`: <class 'str'>
- `thought`: <class 'str'>
- `observation`: <class 'str'>
### Methods
### `__init__(self, uri: str, entity_type: str = '', action: str = '', arguments: str = '', thought: str = '', observation: str = '') -> None`
### `__init__(self, uri: str, entity_type: str = '', action: str = '', arguments: str = '', thought: str = '') -> None`
Initialize self. See help(type(self)) for accurate signature.
---
## `Observation`
```python
from trustgraph.api import Observation
```
Observation entity - standalone tool result (Agent only).
**Fields:**
- `uri`: <class 'str'>
- `entity_type`: <class 'str'>
- `document`: <class 'str'>
### Methods
### `__init__(self, uri: str, entity_type: str = '', document: str = '') -> None`
Initialize self. See help(type(self)) for accurate signature.
@ -3761,10 +3257,11 @@ These chunks show how the agent is thinking about the problem.
- `content`: <class 'str'>
- `end_of_message`: <class 'bool'>
- `chunk_type`: <class 'str'>
- `message_id`: <class 'str'>
### Methods
### `__init__(self, content: str, end_of_message: bool = False, chunk_type: str = 'thought') -> None`
### `__init__(self, content: str, end_of_message: bool = False, chunk_type: str = 'thought', message_id: str = '') -> None`
Initialize self. See help(type(self)) for accurate signature.
@ -3787,10 +3284,11 @@ These chunks show what the agent learned from using tools.
- `content`: <class 'str'>
- `end_of_message`: <class 'bool'>
- `chunk_type`: <class 'str'>
- `message_id`: <class 'str'>
### Methods
### `__init__(self, content: str, end_of_message: bool = False, chunk_type: str = 'observation') -> None`
### `__init__(self, content: str, end_of_message: bool = False, chunk_type: str = 'observation', message_id: str = '') -> None`
Initialize self. See help(type(self)) for accurate signature.
@ -3818,10 +3316,11 @@ its reasoning and tool use.
- `end_of_message`: <class 'bool'>
- `chunk_type`: <class 'str'>
- `end_of_dialog`: <class 'bool'>
- `message_id`: <class 'str'>
### Methods
### `__init__(self, content: str, end_of_message: bool = False, chunk_type: str = 'final-answer', end_of_dialog: bool = False) -> None`
### `__init__(self, content: str, end_of_message: bool = False, chunk_type: str = 'final-answer', end_of_dialog: bool = False, message_id: str = '') -> None`
Initialize self. See help(type(self)) for accurate signature.
@ -3864,7 +3363,7 @@ from trustgraph.api import ProvenanceEvent
Provenance event for explainability.
Emitted during GraphRAG queries when explainable mode is enabled.
Emitted during retrieval queries when explainable mode is enabled.
Each event represents a provenance node created during query processing.
**Fields:**
@ -3872,10 +3371,12 @@ Each event represents a provenance node created during query processing.
- `explain_id`: <class 'str'>
- `explain_graph`: <class 'str'>
- `event_type`: <class 'str'>
- `entity`: <class 'object'>
- `triples`: <class 'list'>
### Methods
### `__init__(self, explain_id: str, explain_graph: str = '', event_type: str = '') -> None`
### `__init__(self, explain_id: str, explain_graph: str = '', event_type: str = '', entity: object = None, triples: list = <factory>) -> None`
Initialize self. See help(type(self)) for accurate signature.

4
docs/tech-specs/agent-explainability.md
View file

@ -219,8 +219,8 @@ TG_ANSWER = TG + "answer"
| `trustgraph-base/trustgraph/provenance/triples.py` | Add TG types to GraphRAG triple builders, add Document RAG triple builders |
| `trustgraph-base/trustgraph/provenance/uris.py` | Add Document RAG URI generators |
| `trustgraph-base/trustgraph/provenance/__init__.py` | Export new types, predicates, and Document RAG functions |
| `trustgraph-base/trustgraph/schema/services/retrieval.py` | Add explain_id and explain_graph to DocumentRagResponse |
| `trustgraph-base/trustgraph/messaging/translators/retrieval.py` | Update DocumentRagResponseTranslator for explainability fields |
| `trustgraph-base/trustgraph/schema/services/retrieval.py` | Add explain_id, explain_graph, and explain_triples to DocumentRagResponse |
| `trustgraph-base/trustgraph/messaging/translators/retrieval.py` | Update DocumentRagResponseTranslator for explainability fields including inline triples |
| `trustgraph-flow/trustgraph/agent/react/service.py` | Add explainability producer + recording logic |
| `trustgraph-flow/trustgraph/retrieval/document_rag/document_rag.py` | Add explainability callback and emit provenance triples |
| `trustgraph-flow/trustgraph/retrieval/document_rag/rag.py` | Add explainability producer and wire up callback |

939
docs/tech-specs/agent-orchestration.md Normal file
View file

@ -0,0 +1,939 @@
# TrustGraph Agent Orchestration — Technical Specification
## Overview
This specification describes the extension of TrustGraph's agent architecture
from a single ReACT execution pattern to a multi-pattern orchestration
model. The existing Pulsar-based self-queuing loop is pattern-agnostic — the
same infrastructure supports ReACT, Plan-then-Execute, Supervisor/Subagent
fan-out, and other execution strategies without changes to the message
transport. The extension adds a routing layer that selects the appropriate
pattern for each task, a set of pattern implementations that share common
iteration infrastructure, and a fan-out/fan-in mechanism for multi-agent
coordination.
The central design principle is that
**trust and explainability are structural properties of the architecture**,
achieved by constraining LLM decisions to
graph-defined option sets and recording those constraints in the execution
trace.
---
## Background
### Existing Architecture
The current agent manager is built on the ReACT pattern (Reasoning + Acting)
with these properties:
- **Self-queuing loop**: Each iteration emits a new Pulsar message carrying
the accumulated history. The agent manager picks this up and runs the next
iteration.
- **Stateless agent manager**: No in-process state. All state lives in the
message payload.
- **Natural parallelism**: Multiple independent agent requests are handled
concurrently across Pulsar consumers.
- **Durability**: Crash recovery is inherent — the message survives process
failure.
- **Real-time feedback**: Streaming thought, action, observation and answer
chunks are emitted as iterations complete.
- **Tool calling and MCP invocation**: Tool calls into knowledge graphs,
external services, and MCP-connected systems.
- **Decision traces written to the knowledge graph**: Every iteration records
PROV-O triples — session, analysis, and conclusion entities — forming the
basis of explainability.
### Current Message Flow
```
AgentRequest arrives (question, history=[], state, group, session_id)
│
▼
Filter tools by group/state
│
▼
AgentManager.react() → LLM call → parse → Action or Final
│ │
│ [Action] │ [Final]
▼ ▼
Execute tool, capture observation Emit conclusion triples
Emit iteration triples Send AgentResponse
Append to history (end_of_dialog=True)
Emit new AgentRequest → "next" topic
│
└── (picked up again by consumer, loop continues)
```
The key insight is that this loop structure is not ReACT-specific. The
plumbing — receive message, do work, emit next message — is the same
regardless of what the "work" step does. The payload and the pattern logic
define the behaviour; the infrastructure remains constant.
### Current Limitations
- Only one execution pattern (ReACT) is available regardless of task
characteristics.
- No mechanism for one agent to spawn and coordinate subagents.
- Pattern selection is implicit — every task gets the same treatment.
- The provenance model assumes a linear iteration chain (analysis N derives
from analysis N-1), with no support for parallel branches.
---
## Design Goals
- **Pattern-agnostic iteration infrastructure**: The self-queuing loop, tool
filtering, provenance emission, and streaming feedback should be shared
across all patterns.
- **Graph-constrained pattern selection**: The LLM selects patterns from a
graph-defined set, not from unconstrained reasoning. This makes the
selection auditable and explainable.
- **Genuinely parallel fan-out**: Subagent tasks execute concurrently on the
Pulsar queue, not sequentially in a single process.
- **Stateless coordination**: Fan-in uses the knowledge graph as coordination
substrate. The agent manager remains stateless.
- **Additive change**: The existing ReACT flow continues to work
unchanged. New patterns are added alongside it, not in place of it.
---
## Patterns
### ReACT as One Pattern Among Many
ReACT is one point in a wider space of agent execution strategies:
| Pattern | Structure | Strengths |
|---|---|---|
| **ReACT** | Interleaved reasoning and action | Adaptive, good for open-ended tasks |
| **Plan-then-Execute** | Decompose into a step DAG, then execute | More predictable, auditable plan |
| **Reflexion** | ReACT + self-critique after each action | Agents improve within the episode |
| **Supervisor/Subagent** | One agent orchestrates others | Parallel decomposition, synthesis |
| **Debate/Ensemble** | Multiple agents reason independently | Diverse perspectives, reconciliation |
| **LLM-as-router** | No reasoning loop, pure dispatch | Fast classification and routing |
Not all of these need to be implemented at once. The architecture should
support them; the initial implementation delivers ReACT (already exists),
Plan-then-Execute, and Supervisor/Subagent.
### Pattern Storage
Patterns are stored as configuration items via the config API. They are
finite in number, mechanically well-defined, have enumerable properties,
and change slowly. Each pattern is a JSON object stored under the
`agent-pattern` config type.
```json
Config type: "agent-pattern"
Config key: "react"
Value: {
"name": "react",
"description": "ReACT — Reasoning + Acting",
"when_to_use": "Adaptive, good for open-ended tasks"
}
```
These are written at deployment time and change rarely. If the architecture
later benefits from graph-based pattern storage (e.g. for richer ontological
relationships), the config items can be migrated to graph nodes — the
meta-router's selection logic is the same regardless of backend.
---
## Task Types
### What a Task Type Represents
A **task type** characterises the problem domain — what the agent is being
asked to accomplish, and how a domain expert would frame it analytically.
- Carries domain-specific methodology (e.g. "intelligence analysis always
applies structured analytic techniques")
- Pre-populates initial reasoning context via a framing prompt
- Constrains which patterns are valid for this class of problem
- Can define domain-specific termination criteria
### Identification
Task types are identified from plain-text task descriptions by the
LLM. Building a formal ontology over task descriptions is premature — natural
language is too varied and context-dependent. The LLM reads the description;
the graph provides the structure downstream.
### Task Type Storage
Task types are stored as configuration items via the config API under the
`agent-task-type` config type. Each task type is a JSON object that
references valid patterns by name.
```json
Config type: "agent-task-type"
Config key: "risk-assessment"
Value: {
"name": "risk-assessment",
"description": "Due Diligence / Risk Assessment",
"framing_prompt": "Analyse across financial, reputational, legal and operational dimensions using structured analytic techniques.",
"valid_patterns": ["supervisor", "plan-then-execute", "react"],
"when_to_use": "Multi-dimensional analysis requiring structured assessment"
}
```
The `valid_patterns` list defines the constrained decision space — the LLM
can only select patterns that the task type's configuration says are valid.
This is the many-to-many relationship between task types and patterns,
expressed as configuration rather than graph edges.
### Selection Flow
```
Task Description (plain text, from AgentRequest.question)
│
│ [LLM interprets, constrained by available task types from config]
▼
Task Type (config item — domain framing and methodology)
│
│ [config lookup — valid_patterns list]
▼
Pattern Candidates (config items)
│
│ [LLM selects within constrained set,
│ informed by task description signals:
│ complexity, urgency, scope]
▼
Selected Pattern
```
The task description may carry modulating signals (complexity, urgency, scope)
that influence which pattern is selected within the constrained set. But the
raw description never directly selects a pattern — it always passes through
the task type layer first.
---
## Explainability Through Constrained Decision Spaces
A central principle of TrustGraph's explainability architecture is that
**explainability comes from constrained decision spaces**.
When a decision is made from an unconstrained space — a raw LLM call with no
guardrails — the reasoning is opaque even if the LLM produces a rationale,
because that rationale is post-hoc and unverifiable.
When a decision is made from a **constrained set defined in configuration**,
you can always answer:
- What valid options were available
- What criteria narrowed the set
- What signal made the final selection within that set
This principle already governs the existing decision trace architecture and
extends naturally to pattern selection. The routing decision — which task type
and which pattern — is itself recorded as a provenance node, making the first
decision in the execution trace auditable.
**Trust becomes a structural property of the architecture, not a claimed
property of the model.**
---
## Orchestration Architecture
### The Meta-Router
The meta-router is the entry point for all agent requests. It runs as a
pre-processing step before the pattern-specific iteration loop begins. Its
job is to determine the task type and select the execution pattern.
**When it runs**: On receipt of an `AgentRequest` with empty history (i.e. a
new task, not a continuation). Requests with non-empty history are already
mid-iteration and bypass the meta-router.
**What it does**:
1. Lists all available task types from the config API
(`config.list("agent-task-type")`).
2. Presents these to the LLM alongside the task description. The LLM
identifies which task type applies (or "general" as a fallback).
3. Reads the selected task type's configuration to get the `valid_patterns`
list.
4. Loads the candidate pattern definitions from config and presents them to
the LLM. The LLM selects one, influenced by signals in the task
description (complexity, number of independent dimensions, urgency).
5. Records the routing decision as a provenance node (see Provenance Model
below).
6. Populates the `AgentRequest` with the selected pattern, task type framing
prompt, and any pattern-specific configuration, then emits it onto the
queue.
**Where it lives**: The meta-router is a phase within the agent-orchestrator,
not a separate service. The agent-orchestrator is a new executable that
uses the same service identity as the existing agent-manager-react, making
it a drop-in replacement on the same Pulsar queues. It includes the full
ReACT implementation alongside the new orchestration patterns. The
distinction between "route" and "iterate" is determined by whether the
request already has a pattern set.
### Pattern Dispatch
Once the meta-router has annotated the request with a pattern, the agent
manager dispatches to the appropriate pattern implementation. This is a
straightforward branch on the pattern field:
```
request arrives
│
├── history is empty → meta-router → annotate with pattern → re-emit
│
└── history is non-empty (or pattern is set)
│
├── pattern = "react" → ReACT iteration
├── pattern = "plan-then-execute" → PtE iteration
├── pattern = "supervisor" → Supervisor iteration
└── (no pattern) → ReACT iteration (default)
```
Each pattern implementation follows the same contract: receive a request, do
one iteration of work, then either emit a "next" message (continue) or emit a
response (done). The self-queuing loop doesn't change.
### Pattern Implementations
#### ReACT (Existing)
No changes. The existing `AgentManager.react()` path continues to work
as-is.
#### Plan-then-Execute
Two-phase pattern:
**Planning phase** (first iteration):
- LLM receives the question plus task type framing.
- Produces a structured plan: an ordered list of steps, each with a goal,
expected tool, and dependencies on prior steps.
- The plan is recorded in the history as a special "plan" step.
- Emits a "next" message to begin execution.
**Execution phase** (subsequent iterations):
- Reads the plan from history.
- Identifies the next unexecuted step.
- Executes that step (tool call + observation), similar to a single ReACT
action.
- Records the result against the plan step.
- If all steps complete, synthesises a final answer.
- If a step fails or produces unexpected results, the LLM can revise the
remaining plan (bounded re-planning, not a full restart).
The plan lives in the history, so it travels with the message. No external
state is needed.
#### Supervisor/Subagent
The supervisor pattern introduces fan-out and fan-in. This is the most
architecturally significant addition.
**Supervisor planning iteration**:
- LLM receives the question plus task type framing.
- Decomposes the task into independent subagent goals.
- For each subagent, emits a new `AgentRequest` with:
- A focused question (the subagent's specific goal)
- A shared correlation ID tying it to the parent task
- The subagent's own pattern (typically ReACT, but could be anything)
- Relevant context sliced from the parent request
**Subagent execution**:
- Each subagent request is picked up by an agent manager instance and runs its
own independent iteration loop.
- Subagents are ordinary agent executions — they self-queue, use tools, emit
provenance, stream feedback.
- When a subagent reaches a Final answer, it writes a completion record to the
knowledge graph under the shared correlation ID.
**Fan-in and synthesis**:
- An aggregator detects when all sibling subagents for a correlation ID have
completed.
- It emits a synthesis request to the supervisor carrying the correlation ID.
- The supervisor queries the graph for subagent results, reasons across them,
and decides whether to emit a final answer or iterate again.
**Supervisor re-iteration**:
- After synthesis, the supervisor may determine that the results are
incomplete, contradictory, or reveal gaps requiring further investigation.
- Rather than emitting a final answer, it can fan out again with new or
refined subagent goals under a new correlation ID. This is the same
self-queuing loop — the supervisor emits new subagent requests and stops,
the aggregator detects completion, and synthesis runs again.
- The supervisor's iteration count (planning + synthesis rounds) is bounded
to prevent unbounded looping.
This is detailed further in the Fan-Out / Fan-In section below.
---
## Message Schema Evolution
### Shared Schema Principle
The `AgentRequest` and `AgentResponse` schemas are the shared contract
between the agent-manager (existing ReACT execution) and the
agent-orchestrator (meta-routing, supervisor, plan-then-execute). Both
services consume from the same *agent request* topic using the same
schema. Any schema changes must be reflected in both — the schema is
the integration point, not the service implementation.
This means the orchestrator does not introduce separate message types for
its own use. Subagent requests, synthesis triggers, and meta-router
outputs are all `AgentRequest` messages with different field values. The
agent-manager ignores orchestration fields it doesn't use.
### New Fields
The `AgentRequest` schema needs new fields to carry orchestration
metadata.
```python
@dataclass
class AgentRequest:
# Existing fields (unchanged)
question: str = ""
state: str = ""
group: list[str] | None = None
history: list[AgentStep] = field(default_factory=list)
user: str = ""
collection: str = "default"
streaming: bool = False
session_id: str = ""
# New orchestration fields
conversation_id: str = "" # Optional caller-generated ID grouping related requests
pattern: str = "" # "react", "plan-then-execute", "supervisor", ""
task_type: str = "" # Identified task type name
framing: str = "" # Task type framing prompt injected into LLM context
correlation_id: str = "" # Shared ID linking subagents to parent
parent_session_id: str = "" # Parent's session_id (for subagents)
subagent_goal: str = "" # Focused goal for this subagent
expected_siblings: int = 0 # How many sibling subagents exist
```
The `AgentStep` schema also extends to accommodate non-ReACT iteration types:
```python
@dataclass
class AgentStep:
# Existing fields (unchanged)
thought: str = ""
action: str = ""
arguments: dict[str, str] = field(default_factory=dict)
observation: str = ""
user: str = ""
# New fields
step_type: str = "" # "react", "plan", "execute", "supervise", "synthesise"
plan: list[PlanStep] | None = None # For plan-then-execute: the structured plan
subagent_results: dict | None = None # For supervisor: collected subagent outputs
```
The `PlanStep` structure for Plan-then-Execute:
```python
@dataclass
class PlanStep:
goal: str = "" # What this step should accomplish
tool_hint: str = "" # Suggested tool (advisory, not binding)
depends_on: list[int] = field(default_factory=list) # Indices of prerequisite steps
status: str = "pending" # "pending", "complete", "failed", "revised"
result: str = "" # Observation from execution
```
---
## Fan-Out and Fan-In
### Why This Matters
Fan-out is the mechanism that makes multi-agent coordination genuinely
parallel rather than simulated. With Pulsar, emitting multiple messages means
multiple consumers can pick them up concurrently. This is not threading or
async simulation — it is real distributed parallelism across agent manager
instances.
### Fan-Out: Supervisor Emits Subagent Requests
When a supervisor iteration decides to decompose a task, it:
1. Generates a **correlation ID** — a UUID that groups the sibling subagents.
2. For each subagent, constructs a new `AgentRequest`:
- `question` = the subagent's focused goal (from `subagent_goal`)
- `correlation_id` = the shared correlation ID
- `parent_session_id` = the supervisor's session_id
- `pattern` = typically "react", but the supervisor can specify any pattern
- `session_id` = a new unique ID for this subagent's own provenance chain
- `expected_siblings` = total number of sibling subagents
- `history` = empty (fresh start, but framing context inherited)
- `group`, `user`, `collection` = inherited from parent
3. Emits each subagent request onto the agent request topic.
4. Records the fan-out decision in the provenance graph (see below).
The supervisor then **stops**. It does not wait. It does not poll. It has
emitted its messages and its iteration is complete. The graph and the
aggregator handle the rest.
### Fan-In: Graph-Based Completion Detection
When a subagent reaches its Final answer, it writes a **completion node** to
the knowledge graph:
```
Completion node:
rdf:type tg:SubagentCompletion
tg:correlationId <shared correlation ID>
tg:subagentSessionId <this subagent's session_id>
tg:parentSessionId <supervisor's session_id>
tg:subagentGoal <what this subagent was asked to do>
tg:result → <document URI in librarian>
prov:wasGeneratedBy → <this subagent's conclusion entity>
```
The **aggregator** is a component that watches for completion nodes. When it
detects that all expected siblings for a correlation ID have written
completion nodes, it:
1. Collects all sibling results from the graph and librarian.
2. Constructs a **synthesis request** — a new `AgentRequest` addressed to the supervisor flow:
- `session_id` = the original supervisor's session_id
- `pattern` = "supervisor"
- `step_type` = "synthesise" (carried in history)
- `subagent_results` = the collected findings
- `history` = the supervisor's history up to the fan-out point, plus the synthesis step
3. Emits this onto the agent request topic.
The supervisor picks this up, reasons across the aggregated findings, and
produces its final answer.
### Aggregator Design
The aggregator is event-driven, consistent with TrustGraph's Pulsar-based
architecture. Polling would be an anti-pattern in a system where all
coordination is message-driven.
**Mechanism**: The aggregator is a Pulsar consumer on the explainability
topic. Subagent completion nodes are emitted as triples on this topic as
part of the existing provenance flow. When the aggregator receives a
`tg:SubagentCompletion` triple, it:
1. Extracts the `tg:correlationId` from the completion node.
2. Queries the graph to count how many siblings for that correlation ID
have completed.
3. If all `expected_siblings` are present, triggers fan-in immediately —
collects results and emits the synthesis request.
**State**: The aggregator is stateless in the same sense as the agent
manager — it holds no essential in-memory state. The graph is the source
of truth for completion counts. If the aggregator restarts, it can
re-process unacknowledged completion messages from Pulsar and re-check the
graph. No coordination state is lost.
**Consistency**: Because the completion check queries the graph rather than
relying on an in-memory counter, the aggregator is tolerant of duplicate
messages, out-of-order delivery, and restarts. The graph query is
idempotent — asking "are all siblings complete?" gives the same answer
regardless of how many times or in what order the events arrive.
### Timeout and Failure
- **Subagent timeout**: The aggregator records the timestamp of the first
sibling completion (from the graph). A periodic timeout check (the one
concession to polling — but over local state, not the graph) detects
stalled correlation IDs. If `expected_siblings` completions are not
reached within a configurable timeout, the aggregator emits a partial
synthesis request with whatever results are available, flagging the
incomplete subagents.
- **Subagent failure**: If a subagent errors out, it writes an error
completion node (with `tg:status = "error"` and an error message). The
aggregator treats this as a completion — the supervisor receives the error
in its synthesis input and can reason about partial results.
- **Supervisor iteration limit**: The supervisor's own iteration count
(planning + synthesis) is bounded by `max_iterations` just like any other
pattern.
---
## Provenance Model Extensions
### Routing Decision
The meta-router's task type and pattern selection is recorded as the first
provenance node in the session:
```
Routing node:
rdf:type prov:Entity, tg:RoutingDecision
prov:wasGeneratedBy → session (Question) activity
tg:taskType → TaskType node URI
tg:selectedPattern → Pattern node URI
tg:candidatePatterns → [Pattern node URIs] (what was available)
tg:routingRationale → document URI in librarian (LLM's reasoning)
```
This captures the constrained decision space: what candidates existed, which
was selected, and why. The candidates are graph-derived; the rationale is
LLM-generated but verifiable against the candidates.
### Fan-Out Provenance
When a supervisor fans out, the provenance records the decomposition:
```
FanOut node:
rdf:type prov:Entity, tg:FanOut
prov:wasDerivedFrom → supervisor's routing or planning iteration
tg:correlationId <correlation ID>
tg:subagentGoals → [document URIs for each subagent goal]
tg:expectedSiblings <count>
```
Each subagent's provenance chain is independent (its own session, iterations,
conclusion) but linked back to the parent via:
```
Subagent session:
rdf:type prov:Activity, tg:Question, tg:AgentQuestion
tg:parentCorrelationId <correlation ID>
tg:parentSessionId <supervisor session URI>
```
### Fan-In Provenance
The synthesis step links back to all subagent conclusions:
```
Synthesis node:
rdf:type prov:Entity, tg:Synthesis
prov:wasDerivedFrom → [all subagent Conclusion entities]
tg:correlationId <correlation ID>
```
This creates a DAG in the provenance graph: the supervisor's routing fans out
to N parallel subagent chains, which fan back in to a synthesis node. The
entire multi-agent execution is traceable from a single correlation ID.
### URI Scheme
Extending the existing `urn:trustgraph:agent:{session_id}` pattern:
| Entity | URI Pattern |
|---|---|
| Session (existing) | `urn:trustgraph:agent:{session_id}` |
| Iteration (existing) | `urn:trustgraph:agent:{session_id}/i{n}` |
| Conclusion (existing) | `urn:trustgraph:agent:{session_id}/answer` |
| Routing decision | `urn:trustgraph:agent:{session_id}/routing` |
| Fan-out record | `urn:trustgraph:agent:{session_id}/fanout/{correlation_id}` |
| Subagent completion | `urn:trustgraph:agent:{session_id}/completion` |
---
## Storage Responsibilities
Pattern and task type definitions live in the config API. Runtime state and
provenance live in the knowledge graph. The division is:
| Role | Storage | When Written | Content |
|---|---|---|---|
| Pattern definitions | Config API | At design time | Pattern properties, descriptions |
| Task type definitions | Config API | At design time | Domain framing, valid pattern lists |
| Routing decision trace | Knowledge graph | At request arrival | Why this task type and pattern were selected |
| Iteration decision trace | Knowledge graph | During execution | Each think/act/observe cycle, per existing model |
| Fan-out coordination | Knowledge graph | During fan-out | Subagent goals, correlation ID, expected count |
| Subagent completion | Knowledge graph | During fan-in | Per-subagent results under shared correlation ID |
| Execution audit trail | Knowledge graph | Post-execution | Full multi-agent reasoning trace as a DAG |
The config API holds the definitions that constrain decisions. The knowledge
graph holds the runtime decisions and their provenance. The fan-in
coordination state is part of the provenance automatically — subagent
completion nodes are both coordination signals and audit trail entries.
---
## Worked Example: Partner Risk Assessment
**Request**: "Assess the risk profile of Company X as a potential partner"
**1. Request arrives** on the *agent request* topic with empty history.
The agent manager picks it up.
**2. Meta-router**:
- Queries config API, finds task types: *Risk Assessment*, *Research*,
*Summarisation*, *General*.
- LLM identifies *Risk Assessment*. Framing prompt loaded: "analyse across
financial, reputational, legal and operational dimensions using structured
analytic techniques."
- Valid patterns for *Risk Assessment*: [*Supervisor/Subagent*,
*Plan-then-Execute*, *ReACT*].
- LLM selects *Supervisor/Subagent* — task has four independent investigative
dimensions, well-suited to parallel decomposition.
- Routing decision written to graph. Request re-emitted on the
*agent request* topic with `pattern="supervisor"`, framing populated.
**3. Supervisor iteration** (picked up from *agent request* topic):
- LLM receives question + framing. Reasons that four independent investigative
threads are required.
- Generates correlation ID `corr-abc123`.
- Emits four subagent requests on the *agent request* topic:
- Financial analysis (pattern="react", subagent_goal="Analyse financial
health and stability of Company X")
- Legal analysis (pattern="react", subagent_goal="Review regulatory filings,
sanctions, and legal exposure for Company X")
- Reputational analysis (pattern="react", subagent_goal="Analyse news
sentiment and public reputation of Company X")
- Operational analysis (pattern="react", subagent_goal="Assess supply chain
dependencies and operational risks for Company X")
- Fan-out node written to graph.
**4. Four subagents run in parallel** (each picked up from the *agent
request* topic by agent manager instances), each as an independent ReACT
loop:
- Financial — queries financial data services and knowledge graph
relationships
- Legal — searches regulatory filings and sanctions lists
- Reputational — searches news, analyses sentiment
- Operational — queries supply chain databases
Each self-queues its iterations on the *agent request* topic. Each writes
its own decision trace to the graph as it progresses. Each completes
independently.
**5. Fan-in**:
- Each subagent writes a `tg:SubagentCompletion` node to the graph on
completion, emitted on the *explainability* topic. The completion node
references the subagent's result document in the librarian.
- Aggregator (consuming the *explainability* topic) sees each completion
event. It queries the graph for the fan-out node to get the expected
sibling count, then checks how many completions exist for
`corr-abc123`.
- When all four siblings are complete, the aggregator emits a synthesis
request on the *agent request* topic with the correlation ID. It does
not fetch or bundle subagent results — the supervisor will query the
graph for those.
**6. Supervisor synthesis** (picked up from *agent request* topic):
- Receives the synthesis trigger carrying the correlation ID.
- Queries the graph for `tg:SubagentCompletion` nodes under
`corr-abc123`, retrieving each subagent's goal and result document
reference.
- Fetches the result documents from the librarian.
- Reasons across all four dimensions, produces a structured risk
assessment with confidence scores.
- Emits final answer on the *agent response* topic and writes conclusion
provenance to the graph.
**7. Response delivered** — the supervisor's synthesis streams on the
*agent response* topic as the LLM generates it, with `end_of_dialog`
on the final chunk. The collated answer is saved to the librarian and
referenced from conclusion provenance in the graph. The graph now holds
a complete, human-readable trace of the entire multi-agent execution —
from pattern selection through four parallel investigations to final
synthesis.
---
## Class Hierarchy
The agent-orchestrator executable (`agent-orchestrator`) uses the same
service identity as agent-manager-react, making it a drop-in replacement.
The pattern dispatch model suggests a class hierarchy where shared iteration
infrastructure lives in a base class and pattern-specific logic is in
subclasses:
```
AgentService (base — Pulsar consumer/producer specs, request handling)
│
└── Processor (agent-orchestrator service)
│
├── MetaRouter — task type identification, pattern selection
│
├── PatternBase — shared: tool filtering, provenance, streaming, history
│ ├── ReactPattern — existing ReACT logic (extract from current AgentManager)
│ ├── PlanThenExecutePattern — plan phase + execute phase
│ └── SupervisorPattern — fan-out, synthesis
│
└── Aggregator — fan-in completion detection
```
`PatternBase` captures what is currently spread across `Processor` and
`AgentManager`: tool filtering, LLM invocation, provenance triple emission,
streaming callbacks, history management. The pattern subclasses implement only
the decision logic specific to their execution strategy — what to do with the
LLM output, when to terminate, whether to fan out.
This refactoring is not strictly necessary for the first iteration — the
meta-router and pattern dispatch could be added as branches within the
existing `Processor.agent_request()` method. But the class hierarchy clarifies
where shared vs. pattern-specific logic lives and will prevent duplication as
more patterns are added.
---
## Configuration
### Config API Seeding
Pattern and task type definitions are stored via the config API and need to
be seeded at deployment time. This is analogous to how flow blueprints and
parameter types are loaded — a bootstrap step that writes the initial
configuration.
The initial seed includes:
**Patterns** (config type `agent-pattern`):
- `react` — interleaved reasoning and action
- `plan-then-execute` — structured plan followed by step execution
- `supervisor` — decomposition, fan-out to subagents, synthesis
**Task types** (config type `agent-task-type`, initial set, expected to grow):
- `general` — no specific domain framing, all patterns valid
- `research` — open-ended investigation, valid patterns: react, plan-then-execute
- `risk-assessment` — multi-dimensional analysis, valid patterns: supervisor,
plan-then-execute, react
- `summarisation` — condense information, valid patterns: react
The seed data is configuration, not code. It can be extended via the config
API (or the configuration UI) without redeploying the agent manager.
### Migration Path
The config API provides a practical starting point. If richer ontological
relationships between patterns, task types, and domain knowledge become
valuable, the definitions can be migrated to graph storage. The meta-router's
selection logic queries an abstract set of task types and patterns — the
storage backend is an implementation detail.
### Fallback Behaviour
If the config contains no patterns or task types:
- Task type defaults to `general`.
- Pattern defaults to `react`.
- The system degrades gracefully to existing behaviour.
---
## Design Decisions
| Decision | Resolution | Rationale |
|---|---|---|
| Task type identification | LLM interprets from plain text | Natural language too varied to formalise prematurely |
| Pattern/task type storage | Config API initially, graph later if needed | Avoids graph model complexity upfront; config API already has UI support; migration path is straightforward |
| Meta-router location | Phase within agent manager, not separate service | Avoids an extra network hop; routing is fast |
| Fan-in mechanism | Event-driven via explainability topic | Consistent with Pulsar-based architecture; graph query for completion count is idempotent and restart-safe |
| Aggregator deployment | Separate lightweight process | Decoupled from agent manager lifecycle |
| Subagent pattern selection | Supervisor specifies per-subagent | Supervisor has task context to make this choice |
| Plan storage | In message history | No external state needed; plan travels with message |
| Default pattern | Empty pattern field → ReACT | Sensible default when meta-router is not configured |
---
## Streaming Protocol
### Current Model
The existing agent response schema has two levels:
- **`end_of_message`** — marks the end of a complete thought, observation,
or answer. Chunks belonging to the same message arrive sequentially.
- **`end_of_dialog`** — marks the end of the entire agent execution. No
more messages will follow.
This works because the current system produces messages serially — one
thought at a time, one agent at a time.
### Fan-Out Breaks Serial Assumptions
With supervisor/subagent fan-out, multiple subagents stream chunks
concurrently on the same *agent response* topic. The caller receives
interleaved chunks from different sources and needs to demultiplex them.
### Resolution: Message ID
Each chunk carries a `message_id` — a per-message UUID generated when
the agent begins streaming a new thought, observation, or answer. The
caller groups chunks by `message_id` and assembles each message
independently.
```
Response chunk fields:
message_id UUID for this message (groups chunks)
session_id Which agent session produced this chunk
chunk_type "thought" | "observation" | "answer" | ...
content The chunk text
end_of_message True on the final chunk of this message
end_of_dialog True on the final message of the entire execution
```
A single subagent emits multiple messages (thought, observation, thought,
answer), each with a distinct `message_id`. The `session_id` identifies
which subagent the message belongs to. The caller can display, group, or
filter by either.
### Provenance Trigger
`end_of_message` is the trigger for provenance storage. When a complete
message has been assembled from its chunks:
1. The collated text is saved to the librarian as a single document.
2. A provenance node is written to the graph referencing the document URI.
This follows the pattern established by GraphRAG, where streaming synthesis
chunks are delivered live but the stored provenance references the collated
answer text. Streaming is for the caller; provenance needs complete messages.
---
## Open Questions
- **Re-planning depth** (resolved): Runtime parameter on the
agent-orchestrator executable, default 2. Bounds how many times
Plan-then-Execute can revise its plan before forcing termination.
- **Nested fan-out** (phase B): A subagent can itself be a supervisor
that fans out further. The architecture supports this — correlation IDs
are independent and the aggregator is stateless. The protocols and
message schema should not preclude nested fan-out, but implementation
is deferred. Depth limits will need to be enforced to prevent runaway
decomposition.
- **Task type evolution** (resolved): Manually curated for now. See
Future Directions below for automated discovery.
- **Cost attribution** (deferred): Costs are measured at the
text-completion queue level as they are today. Per-request attribution
across subagents is not yet implemented and is not a blocker for
orchestration.
- **Conversation ID** (resolved): An optional `conversation_id` field on
`AgentRequest`, generated by the caller. When present, all objects
created during the execution (provenance nodes, librarian documents,
subagent completion records) are tagged with the conversation ID. This
enables querying all interactions in a conversation with a single
lookup, and provides the foundation for conversation-scoped memory.
No explicit open/close — the first request with a new conversation ID
implicitly starts the conversation. Omit for one-shot queries.
- **Tool scoping per subagent** (resolved): Subagents inherit the
parent's tool group by default. The supervisor can optionally override
the group per subagent to constrain capabilities (e.g. financial
subagent gets only financial tools). The `group` field on
`AgentRequest` already supports this — the supervisor just sets it
when constructing subagent requests.
---
## Future Directions
### Automated Task Type Discovery
Task types are manually curated in the initial implementation. However,
the architecture is well-suited to automated discovery because all agent
requests and their execution traces flow through Pulsar topics. A
learning service could consume these messages and analyse patterns in
how tasks are framed, which patterns are selected, and how successfully
they execute. Over time, it could propose new task types based on
clusters of similar requests that don't map well to existing types, or
suggest refinements to framing prompts based on which framings lead to
better outcomes. This service would write proposed task types to the
config API for human review — automated discovery, manual approval. The
agent-orchestrator does not need to change; it always reads task types
from config regardless of how they got there.

282
docs/tech-specs/config-push-poke.md Normal file
View file

@ -0,0 +1,282 @@
# Config Push "Notify" Pattern Technical Specification
## Overview
Replace the current config push mechanism — which broadcasts the full config
blob on a `state` class queue — with a lightweight "notify" notification
containing only the version number and affected types. Processors that care
about those types fetch the full config via the existing request/response
interface.
This solves the RabbitMQ late-subscriber problem: when a process restarts,
its fresh queue has no historical messages, so it never receives the current
config state. With the notify pattern, the push queue is only a signal — the
source of truth is the config service's request/response API, which is
always available.
## Problem
On Pulsar, `state` class queues are persistent topics. A new subscriber
with `InitialPosition.Earliest` reads from message 0 and receives the
last config push. On RabbitMQ, each subscriber gets a fresh per-subscriber
queue (named with a new UUID). Messages published before the queue existed
are gone. A restarting processor never gets the current config.
## Design
### The Notify Message
The `ConfigPush` schema changes from carrying the full config to carrying
just a version number and the list of affected config types:
```python
@dataclass
class ConfigPush:
version: int = 0
types: list[str] = field(default_factory=list)
```
When the config service handles a `put` or `delete`, it knows which types
were affected (from the request's `values[].type` or `keys[].type`). It
includes those in the notify. On startup, the config service sends a notify
with an empty types list (meaning "everything").
### Subscribe-then-Fetch Startup (No Race Condition)
The critical ordering to avoid missing an update:
1. **Subscribe** to the config push queue. Buffer incoming notify messages.
2. **Fetch** the full config via request/response (`operation: "config"`).
This returns the config dict and a version number.
3. **Apply** the fetched config to all registered handlers.
4. **Process** buffered notifys. For any notify with `version > fetched_version`,
re-fetch and re-apply. Discard notifys with `version <= fetched_version`.
5. **Enter steady state**. Process future notifys as they arrive.
This is safe because:
- If an update happens before the subscription, the fetch picks it up.
- If an update happens between subscribe and fetch, it's in the buffer.
- If an update happens after the fetch, it arrives on the queue normally.
- Version comparison ensures no duplicate processing.
### Processor API
The current API requires processors to understand the full config dict
structure. The new API should be cleaner — processors declare which config
types they care about and provide a handler that receives only the relevant
config subset.
#### Current API
```python
# In processor __init__:
self.register_config_handler(self.on_configure_flows)
# Handler receives the entire config dict:
async def on_configure_flows(self, config, version):
if "active-flow" not in config:
return
if self.id in config["active-flow"]:
flow_config = json.loads(config["active-flow"][self.id])
# ...
```
#### New API
```python
# In processor __init__:
self.register_config_handler(
handler=self.on_configure_flows,
types=["active-flow"],
)
# Handler receives only the relevant config subset, same signature:
async def on_configure_flows(self, config, version):
# config still contains the full dict, but handler is only called
# when "active-flow" type changes (or on startup)
if "active-flow" not in config:
return
# ...
```
The `types` parameter is optional. If omitted, the handler is called for
every config change (backward compatible). If specified, the handler is
only invoked when the notify's `types` list intersects with the handler's
types, or on startup (empty types list = everything).
#### Internal Registration Structure
```python
# In AsyncProcessor:
def register_config_handler(self, handler, types=None):
self.config_handlers.append({
"handler": handler,
"types": set(types) if types else None, # None = all types
})
```
#### Notify Processing Logic
```python
async def on_config_notify(self, message, consumer, flow):
notify_version = message.value().version
notify_types = set(message.value().types)
# Skip if we already have this version or newer
if notify_version <= self.config_version:
return
# Fetch full config from config service
config, version = await self.config_client.config()
self.config_version = version
# Determine which handlers to invoke
for entry in self.config_handlers:
handler_types = entry["types"]
if handler_types is None:
# Handler cares about everything
await entry["handler"](config, version)
elif not notify_types or notify_types & handler_types:
# notify_types empty = startup (invoke all),
# or intersection with handler's types
await entry["handler"](config, version)
```
### Config Service Changes
#### Push Method
The `push()` method changes to send only version + types:
```python
async def push(self, types=None):
version = await self.config.get_version()
resp = ConfigPush(
version=version,
types=types or [],
)
await self.config_push_producer.send(resp)
```
#### Put/Delete Handlers
Extract affected types and pass to push:
```python
async def handle_put(self, v):
types = list(set(k.type for k in v.values))
for k in v.values:
await self.table_store.put_config(k.type, k.key, k.value)
await self.inc_version()
await self.push(types=types)
async def handle_delete(self, v):
types = list(set(k.type for k in v.keys))
for k in v.keys:
await self.table_store.delete_key(k.type, k.key)
await self.inc_version()
await self.push(types=types)
```
#### Queue Class Change
The config push queue changes from `state` class to `flow` class. The push
is now a transient signal — the source of truth is the config service's
request/response API, not the queue. `flow` class is persistent (survives
broker restarts) but doesn't require last-message retention, which was the
root cause of the RabbitMQ problem.
```python
config_push_queue = queue('config', cls='flow') # was cls='state'
```
#### Startup Push
On startup, the config service sends a notify with empty types list
(signalling "everything changed"):
```python
async def start(self):
await self.push(types=[]) # Empty = all types
await self.config_request_consumer.start()
```
### AsyncProcessor Changes
The `AsyncProcessor` needs a config request/response client alongside the
push consumer. The startup sequence becomes:
```python
async def start(self):
# 1. Start the push consumer (begins buffering notifys)
await self.config_sub_task.start()
# 2. Fetch current config via request/response
config, version = await self.config_client.config()
self.config_version = version
# 3. Apply to all handlers (startup = all handlers invoked)
for entry in self.config_handlers:
await entry["handler"](config, version)
# 4. Buffered notifys are now processed by on_config_notify,
# which skips versions <= self.config_version
```
The config client needs to be created in `__init__` using the existing
request/response queue infrastructure. The `ConfigClient` from
`trustgraph.clients.config_client` already exists but uses a synchronous
blocking pattern. An async variant or integration with the processor's
pub/sub backend is needed.
### Existing Config Handler Types
For reference, the config types currently used by handlers:
| Handler | Type(s) | Used By |
|---------|---------|---------|
| `on_configure_flows` | `active-flow` | All FlowProcessor subclasses |
| `on_collection_config` | `collection` | Storage services (triples, embeddings, rows) |
| `on_prompt_config` | `prompt` | Prompt template service, agent extract |
| `on_schema_config` | `schema` | Rows storage, row embeddings, NLP query, structured diag |
| `on_cost_config` | `token-costs` | Metering service |
| `on_ontology_config` | `ontology` | Ontology extraction |
| `on_librarian_config` | `librarian` | Librarian service |
| `on_mcp_config` | `mcp-tool` | MCP tool service |
| `on_knowledge_config` | `kg-core` | Cores service |
## Implementation Order
1. **Update ConfigPush schema** — change `config` field to `types` field.
2. **Update config service** — modify `push()` to send version + types.
Modify `handle_put`/`handle_delete` to extract affected types.
3. **Add async config query to AsyncProcessor** — create a
request/response client for config queries within the processor's
event loop.
4. **Implement subscribe-then-fetch startup** — reorder
`AsyncProcessor.start()` to subscribe first, then fetch, then
process buffered notifys with version comparison.
5. **Update register_config_handler** — add optional `types` parameter.
Update `on_config_notify` to filter by type intersection.
6. **Update existing handlers** — add `types` parameter to all
`register_config_handler` calls across the codebase.
7. **Backward compatibility** — handlers without `types` parameter
continue to work (invoked for all changes).
## Risks
- **Thundering herd**: if many processors restart simultaneously, they
all hit the config service API at once. Mitigated by the config service
already being designed for request/response load, and the number of
processors being small (tens, not thousands).
- **Config service availability**: processors now depend on the config
service being up at startup, not just having received a push. This is
already the case in practice — without config, processors can't do
anything useful.

551
docs/tech-specs/pubsub-abstraction.md Normal file
View file

@ -0,0 +1,551 @@
# Pub/Sub Abstraction: Broker-Independent Messaging
## Problem
TrustGraph's messaging infrastructure is deeply coupled to Apache Pulsar in ways that go beyond the transport layer. This coupling creates several concrete problems.
### 1. Schema system is Pulsar-native
Every message type in the system is defined as a `pulsar.schema.Record` subclass using Pulsar field types (`String()`, `Integer()`, `Boolean()`, etc.). This means:
- The `pulsar` Python package is a build dependency for `trustgraph-base`, even though `trustgraph-base` contains no transport logic
- Any code that imports a message schema transitively depends on Pulsar
- The schema definitions cannot be reused with a different broker without the Pulsar library installed
- What's actually happening on the wire is JSON serialisation — the Pulsar schema machinery adds complexity without adding value over plain JSON encode/decode
### 2. Translators are named after the broker
The translator layer that converts between internal Python objects and wire format uses methods called `to_pulsar()` and `from_pulsar()`. These are really just JSON encode/decode operations — they have nothing to do with Pulsar specifically. The naming creates a false impression that the translation is broker-specific, when in reality any broker that carries JSON payloads would use identical logic.
### 3. Queue names use Pulsar URI format
Queue identifiers throughout the codebase use Pulsar's `persistent://tenant/namespace/topic` or `non-persistent://tenant/namespace/topic` URI format. These are hardcoded in schema definitions and referenced across services. RabbitMQ, Redis Streams, or any other broker would use completely different naming conventions. There is no abstraction between the logical identity of a queue and its broker-specific address.
### 4. Broker selection is not configurable
There is no mechanism to select a different pub/sub backend at deployment time. The Pulsar client is instantiated directly in the gateway and via `PulsarClient` in the base processor. Switching to a different broker would require code changes across multiple packages, not a configuration change.
### 5. Architectural requirements are implicit
TrustGraph relies on specific pub/sub behaviours — shared subscriptions for load balancing, message acknowledgement for reliability, message properties for correlation — but these requirements are not documented. This makes it difficult to evaluate whether a candidate broker (RabbitMQ, Redis Streams, NATS, etc.) actually satisfies the system's needs, or where the gaps would be.
## Design Goals
### Goal 1: Remove the link between Pulsar schemas and application code
Message types should be plain Python objects (dataclasses) that know how to serialise to and from JSON. The `pulsar.schema.Record` base class and Pulsar field types should not appear in schema definitions. The pub/sub transport layer sends and receives JSON bytes; the schema layer handles the mapping between JSON and typed Python objects independently.
### Goal 2: Remove `to_pulsar` / `from_pulsar` naming
The translator methods should reflect what they actually do: encode a Python object to a JSON-compatible dict, and decode a JSON-compatible dict back to a Python object. The naming should be broker-neutral (e.g. `encode` / `decode`, or `to_dict` / `from_dict`).
### Goal 3: Schema objects provide encode/decode
Each message type should be a Python dataclass (or similar) with a well-defined mapping to and from JSON. For example:
```python
@dataclass
class TextCompletionRequest:
system: str
prompt: str
streaming: bool = False
```
Given `{"system": "You are helpful", "prompt": "Hello", "streaming": false}` on the wire, decoding produces an object where `request.system` is `"You are helpful"`, `request.prompt` is `"Hello"`, and `request.streaming` is `False`. Encoding does the reverse. This is the schema's concern, not the broker's.
### Goal 4: Abstract queue naming
Queue identifiers should not use Pulsar URI format (`persistent://tg/flow/topic`). A broker-neutral naming scheme is needed so that each backend can map logical queue names to its native format. The right approach here is not yet clear and needs to be worked through — considerations include how to express quality-of-service, multi-tenancy, and namespace separation without leaking broker concepts.
### Goal 5: Document pub/sub architectural requirements
TrustGraph's actual requirements from the pub/sub layer need to be formally specified. This includes:
- **Delivery semantics**: Which queues need at-least-once delivery? Are any fire-and-forget?
- **Consumer patterns**: Shared subscriptions (competing consumers for load balancing), exclusive subscriptions, fan-out/broadcast
- **Message acknowledgement**: Positive ack, negative ack (redelivery), timeout-based redelivery
- **Message properties**: Key-value metadata on messages used for correlation (e.g. request IDs, flow routing)
- **Ordering guarantees**: Per-topic ordering, per-key ordering, or no ordering required
- **Message size**: Typical and maximum message sizes (some payloads include base64-encoded documents)
- **Persistence**: Which messages must survive broker restarts
- **Consumer positioning**: Ability to consume from earliest (replay) vs latest (live tail)
- **Connection model**: Long-lived connections with reconnection, or transient
Documenting these requirements makes it possible to evaluate RabbitMQ or any other candidate against concrete criteria rather than discovering gaps during implementation.
## Pub/Sub Architectural Requirements (As-Is)
This section documents what TrustGraph currently needs from its pub/sub layer. These are the as-is requirements — some may be revisited or relaxed in a future design if it makes broker portability easier.
### Consumer model
All consumers use **shared subscriptions** (competing consumers). Multiple instances of the same processor read from the same subscription, and each message is delivered to exactly one instance. This is the load-balancing mechanism.
No exclusive or failover subscriptions are used anywhere in the codebase, despite infrastructure support for them.
Consumers support configurable concurrency — multiple async tasks within a single process can independently call `receive()` on the same subscription.
### Delivery semantics
Almost all queues are **non-persistent / best-effort (q0)**. The only persistent queue is `config_push_queue` (q2, exactly-once), which pushes full configuration state to processors. Since config pushes are idempotent (full state, not deltas), the persistence requirement here is about surviving broker restarts, not about exactly-once semantics per se.
Flow processing queues (request/response pairs for LLM, RAG, agent, etc.) are all non-persistent. Messages in flight are lost on broker restart. This is acceptable because:
- Requests originate from a client that will time out and retry
- There is no durable work-in-progress that would be corrupted by message loss
- The system is designed for real-time query processing, not batch pipelines
### Message acknowledgement
**Positive acknowledgement**: After successful handler execution, the message is acknowledged. This removes it from the subscription.
**Negative acknowledgement**: On handler failure (unhandled exception or rate-limit timeout), the message is negatively acknowledged, which triggers redelivery by the broker. Rate-limited messages retry for up to 7200 seconds before giving up and negatively acknowledging.
**Orphaned messages**: In the request-response subscriber pattern, messages that arrive with no matching waiter (e.g. the requester timed out) are positively acknowledged and discarded. This prevents redelivery storms.
### Message properties
Messages carry a small set of key-value string properties as metadata, separate from the payload. The primary use is a `"id"` property for request-response correlation — the requester generates a unique ID, attaches it as a property, and the responder echoes it back so the subscriber can match responses to waiters.
Agent orchestration correlation (`correlation_id`, `parent_session_id`) is carried in the message payload, not in properties.
### Consumer positioning
Two modes are used:
- **Earliest**: The configuration consumer starts from the beginning of the topic to receive full configuration history on startup. This is the only use of earliest positioning.
- **Latest** (default): All flow consumers start from the current position, processing only new messages.
### Message ordering
**Not required.** The codebase explicitly does not depend on message ordering:
- Shared subscriptions distribute messages across consumers without ordering guarantees
- Concurrent handler tasks within a consumer process messages in arbitrary order
- Request-response correlation uses IDs, not positional ordering
- The supervisor fan-out/fan-in pattern collects results in a dictionary, order-independent
- Configuration pushes are full state snapshots, not ordered deltas
### Message sizes
Most messages are small JSON payloads (< 10KB). The exceptions:
- **Document content**: Large documents (PDFs, text files) can be sent through the chunking service with base64 encoding. Pulsar's chunking feature (`chunking_enabled`) handles automatic splitting of oversized messages.
- **Agent observations**: LLM-generated text can be several KB but rarely exceeds typical message size limits.
A replacement broker needs to either support large messages natively or provide a chunking/streaming mechanism. Alternatively, the large-document path could be refactored to use a side-channel (e.g. object store reference) instead of inline payload.
### Fan-out patterns
**Supervisor fan-out**: One supervisor request decomposes into N independent sub-agent requests, each emitted as a separate message on the agent request queue. Different agent instances pick them up via the shared subscription. A correlation ID links the completions back to the original decomposition. This is not pub/sub fan-out (one message to many consumers) — it's application-level fan-out (many messages to one queue).
**Request-response isolation**: Each client creates a unique subscription name on response queues so it only receives its own responses. This means the response queue effectively has many independent subscribers, each seeing a filtered subset of messages based on the `"id"` property match.
### Reconnection and resilience
Reconnection logic lives in the Consumer/Producer/Publisher/Subscriber classes, not in the broker client. These classes handle:
- Automatic reconnection on connection loss
- Retry loops with backoff
- Graceful shutdown (unsubscribe, close)
The broker client itself is expected to provide a basic connection that can fail, and the wrapper classes handle recovery. This is important for the abstraction — the backend interface can be simple because resilience is handled above it.
### Queue inventory
| Queue | Persistence | Purpose |
|-------|-------------|---------|
| config push | Persistent (q2) | Full configuration state broadcast |
| config request/response | Non-persistent | Configuration queries |
| flow request/response | Non-persistent | Flow management |
| knowledge request/response | Non-persistent | Knowledge graph operations |
| librarian request/response | Non-persistent | Document storage operations |
| document embeddings request/response | Non-persistent | Document vector queries |
| row embeddings request/response | Non-persistent | Row vector queries |
| collection request/response | Non-persistent | Collection management |
Additionally, each processing service (LLM, RAG, agent, prompt, embeddings, etc.) has dynamically defined request/response queue pairs configured at deployment time.
### Summary of hard requirements for a replacement broker
1. **Shared subscription / competing consumers** — multiple consumers on one queue, each message delivered to exactly one
2. **Message acknowledgement** — positive ack (remove from queue) and negative ack (trigger redelivery)
3. **Message properties** — key-value metadata on messages, at minimum a string `"id"` field
4. **Two consumer start positions** — from beginning of topic and from current position
5. **Persistence for at least one queue** — config state must survive broker restart
6. **Messages up to several MB** — or a chunking mechanism for large payloads
7. **No ordering requirement** — simplifies broker selection significantly
## Candidate Brokers
A quick assessment of alternatives against the hard requirements above.
### RabbitMQ
The primary candidate. Mature, widely deployed, well understood.
- **Competing consumers**: Yes — multiple consumers on a queue, round-robin delivery. This is RabbitMQ's native model.
- **Acknowledgement**: Yes — `basic.ack` and `basic.nack` with requeue flag.
- **Message properties**: Yes — headers and properties on every message. The `correlation_id` and `message_id` fields are first-class concepts.
- **Consumer positioning**: Yes, via RabbitMQ Streams (3.9+). Streams are append-only logs that support reading from any offset — beginning, end, or timestamp. Classic queues are consumed destructively (no replay), but streams solve this cleanly. The `state` queue class maps to a RabbitMQ stream. Additionally, the Last Value Cache Exchange plugin can retain the most recent message per routing key for new consumers.
- **Persistence**: Yes — durable queues and persistent messages survive broker restart.
- **Large messages**: No hard limit but not designed for very large payloads. Practical limit around 128MB with default config. Adequate for current use.
- **Ordering**: FIFO per queue (stronger than required).
- **Operational complexity**: Low. Single binary, no ZooKeeper/BookKeeper dependencies. Significantly simpler to operate than Pulsar.
- **Ecosystem**: Excellent client libraries, management UI, mature tooling.
**Gaps**: None significant. RabbitMQ Streams cover the replay/earliest positioning requirement.
### Apache Kafka
High-throughput distributed log. More infrastructure than TrustGraph likely needs.
- **Competing consumers**: Yes — consumer groups with partition assignment.
- **Acknowledgement**: Yes — offset commits. No per-message negative ack; failed messages require application-level retry or dead-letter handling.
- **Message properties**: Yes — message headers (key-value byte arrays).
- **Consumer positioning**: Yes — seek to earliest or latest offset. Supports full replay.
- **Persistence**: Yes — all messages are persisted to the log by default.
- **Large messages**: Configurable (`max.message.bytes`), default 1MB, can be increased. Large payloads are discouraged by design.
- **Ordering**: Per-partition ordering (stronger than required).
- **Operational complexity**: High. Requires ZooKeeper (or KRaft), partition management, replication config. Overkill for typical TrustGraph deployments.
- **Ecosystem**: Excellent client libraries, schema registry, Connect framework.
**Gaps**: No native negative acknowledgement. Operational complexity is high for small-to-medium deployments. Partition count must be planned upfront for parallelism.
### Redis Streams
Lightweight option using Redis as a message broker.
- **Competing consumers**: Yes — consumer groups with `XREADGROUP`.
- **Acknowledgement**: Yes — `XACK`. Pending entries list tracks unacknowledged messages. No explicit negative ack but unacknowledged messages can be claimed after timeout via `XAUTOCLAIM`.
- **Message properties**: No native separation between properties and payload. Would need to encode properties as fields within the stream entry or in the payload.
- **Consumer positioning**: Yes — `0` (earliest) or `$` (latest) on group creation.
- **Persistence**: Yes — Redis persistence (RDB/AOF), though Redis is primarily an in-memory system.
- **Large messages**: Practical limit tied to Redis memory. Not suited for large payloads.
- **Ordering**: Per-stream ordering (stronger than required).
- **Operational complexity**: Low if Redis is already in the stack. No additional infrastructure.
**Gaps**: No native message properties. Memory-bound. Persistence depends on Redis configuration. Not a natural fit for message broker patterns.
### NATS / NATS JetStream
Lightweight, high-performance messaging. JetStream adds persistence.
- **Competing consumers**: Yes — queue groups in core NATS; consumer groups in JetStream.
- **Acknowledgement**: JetStream only — `Ack`, `Nak` (with redelivery), `InProgress` (extend timeout).
- **Message properties**: Yes — message headers (key-value).
- **Consumer positioning**: JetStream — deliver all, deliver last, deliver new, deliver by sequence/time.
- **Persistence**: JetStream only. Core NATS is fire-and-forget.
- **Large messages**: Default 1MB, configurable up to 64MB.
- **Ordering**: Per-subject ordering.
- **Operational complexity**: Very low. Single binary, no dependencies. Clustering is straightforward.
**Gaps**: Requires JetStream for persistence and acknowledgement. Smaller ecosystem than RabbitMQ/Kafka.
### Assessment Summary
| Requirement | RabbitMQ | Kafka | Redis Streams | NATS JetStream |
|---|---|---|---|---|
| Competing consumers | Yes | Yes | Yes | Yes |
| Positive/negative ack | Yes | Partial | Partial | Yes |
| Message properties | Yes | Yes | No | Yes |
| Earliest positioning | Yes (Streams) | Yes | Yes | Yes |
| Persistence | Yes | Yes | Partial | Yes |
| Large messages | Yes | Configurable | No | Configurable |
| Operational simplicity | Good | Poor | Good | Good |
**RabbitMQ** is the strongest candidate given TrustGraph's requirements and deployment profile. The only gap (earliest consumer positioning for config) has known workarounds. Operational simplicity is a significant advantage over Pulsar.
## Approach
### Current state
The codebase has already undergone a partial abstraction. The picture is better than the problem statement might suggest:
- **Backend abstraction exists**: `backend.py` defines Protocol-based interfaces (`PubSubBackend`, `BackendProducer`, `BackendConsumer`, `Message`). The Pulsar implementation lives in `pulsar_backend.py`.
- **Schemas are already dataclasses**: Message types in `schema/services/*.py` are plain Python dataclasses with type hints, not Pulsar `Record` subclasses. This was the hardest part of the old spec and it's done.
- **Serialization is JSON-based**: `pulsar_backend.py` contains `dataclass_to_dict()` and `dict_to_dataclass()` helpers that handle the round-trip. The wire format is JSON.
- **Factory pattern exists**: `pubsub.py` has `get_pubsub()` which creates a backend from configuration. Currently only Pulsar is implemented.
- **Consumer/Producer/Publisher/Subscriber are backend-agnostic**: These classes accept a `backend` parameter and delegate transport operations to it. They own retry, reconnection, metrics, and concurrency.
What remains is cleanup, not a rewrite.
### What needs to change
#### 1. Rename translator methods
The translator base class (`messaging/translators/base.py`) defines `to_pulsar()` and `from_pulsar()` as abstract methods. Every translator implements these. The methods convert between external API dicts and internal dataclass objects — nothing Pulsar-specific happens in them.
**Change**: Rename to `decode()` (external dict → dataclass) and `encode()` (dataclass → external dict). Update all translator subclasses and all call sites.
This is a mechanical rename. The method bodies don't change.
#### 2. Rename translator base classes
The base classes `Translator`, `MessageTranslator`, and `SendTranslator` reference "pulsar" in docstrings and parameter names. Clean these up so the naming reflects what the layer actually does: translating between the external API representation (JSON dicts from HTTP/WebSocket) and the internal schema (dataclasses).
#### 3. Move serialization out of the Pulsar backend
`dataclass_to_dict()` and `dict_to_dataclass()` currently live in `pulsar_backend.py` but are not Pulsar-specific. They handle the conversion between dataclasses and JSON-compatible dicts, which every backend needs.
**Change**: Move these to a shared location (e.g. `trustgraph/base/serialization.py` or alongside the schema definitions). The backend interface sends and receives dicts; serialization to/from dataclasses happens at a layer above.
This means the backend Protocol simplifies: `send()` accepts a dict and properties, `value()` returns a dict. The Consumer/Producer layer handles dataclass ↔ dict conversion using the shared serializers.
#### 4. Abstract queue naming
Queue names currently use the format `q0/tg/flow/queue-name` or `q2/tg/config/queue-name`, which the Pulsar backend maps to `non-persistent://tg/flow/queue-name` or `persistent://tg/config/queue-name`.
This is an open design question. Options:
**Option A: Simple string names.** Queues are just strings like `"text-completion-request"`. The backend is responsible for mapping to its native format (Pulsar adds `persistent://tg/flow/` prefix, RabbitMQ uses the string as-is or adds a vhost prefix). Persistence and namespace are configuration concerns, not embedded in the name.
**Option B: Structured queue descriptor.** A small object that carries the logical name plus metadata:
```python
@dataclass
class QueueDescriptor:
name: str # e.g. "text-completion-request"
namespace: str = "flow" # logical grouping
persistent: bool = False # must survive broker restart
```
The backend maps this to its native format.
**Option C: Keep the current format** (`q0/tg/flow/name`) but document it as a TrustGraph convention, not a Pulsar convention. Backends parse it.
Option B is the most explicit. Option A is the simplest. Either is workable. The key constraint is that persistence is a property of the queue definition, not a runtime choice — the config push queue is persistent, everything else is not.
#### 5. Implement RabbitMQ backend
Write `rabbitmq_backend.py` implementing the `PubSubBackend` Protocol:
- **`create_producer()`**: Creates a channel and declares the target queue. `send()` publishes to the default exchange with the queue name as routing key. Properties map to AMQP basic properties (specifically `message_id` for the `"id"` property).
- **`create_consumer()`**: Declares the queue and starts consuming with `basic_consume`. Shared subscription is the default RabbitMQ model — multiple consumers on one queue get round-robin delivery. `acknowledge()` maps to `basic_ack`, `negative_acknowledge()` maps to `basic_nack` with `requeue=True`.
- **Persistence**: For persistent queues, declare as durable with `delivery_mode=2` on messages. For non-persistent queues, declare as non-durable.
- **Consumer positioning**: RabbitMQ queues are consumed destructively, so "earliest" doesn't apply in the Pulsar sense. For the config push use case, use a **fanout exchange with per-consumer exclusive queues** — each new processor gets its own queue that receives all config publishes, plus the last-value can be handled by having the config service re-publish on startup.
- **Large messages**: RabbitMQ handles messages up to `rabbit.max_message_size` (default 128MB). No chunking needed.
The factory in `pubsub.py` gets a new branch:
```python
if backend_type == 'rabbitmq':
return RabbitMQBackend(
host=config.get('rabbitmq_host'),
port=config.get('rabbitmq_port'),
username=config.get('rabbitmq_username'),
password=config.get('rabbitmq_password'),
vhost=config.get('rabbitmq_vhost', '/'),
)
```
Backend selection via `PUBSUB_BACKEND=rabbitmq` environment variable or `--pubsub-backend rabbitmq` CLI flag.
#### 6. Clean up remaining Pulsar references
After the above changes, Pulsar-specific code should be confined to:
- `pulsar_backend.py` — the Pulsar implementation
- `pubsub.py` — the factory that imports it
Audit and remove any remaining Pulsar imports, Pulsar exception handling, or Pulsar-specific concepts from:
- `async_processor.py` (currently catches `_pulsar.Interrupted`)
- `consumer.py`, `subscriber.py` (if any Pulsar exceptions leak through)
- Schema files (should be clean already, but verify)
- Gateway service (currently instantiates Pulsar client directly)
The gateway is a special case — it currently bypasses the abstraction layer and creates a Pulsar client directly for dispatching API requests. It should use the same `get_pubsub()` factory as everything else.
### What stays the same
- **Schema definitions**: Already dataclasses. No changes needed.
- **Consumer/Producer/Publisher/Subscriber**: Already backend-agnostic. No changes to their core logic.
- **FlowProcessor and spec wiring**: Already uses `processor.pubsub` to create backend instances. No changes.
- **Backend Protocol**: The interface in `backend.py` is sound. Minor refinement possible (dict vs dataclass at the boundary) but the shape is right.
### Concrete cleanups
The following files have Pulsar-specific imports that should not be there after the abstraction is complete. Pulsar imports should be confined to `pulsar_backend.py` and the factory in `pubsub.py`.
**Dead imports (unused, can just be removed):**
- `trustgraph-base/trustgraph/base/pubsub.py` — `from pulsar.schema import JsonSchema`, `import pulsar`, `import _pulsar`. The `JsonSchema` import is unused since the switch to `BytesSchema`. The `pulsar`/`_pulsar` imports are only used by the legacy `PulsarClient` class which should be removed (superseded by `PulsarBackend`).
- `trustgraph-base/trustgraph/base/flow_processor.py` — `from pulsar.schema import JsonSchema`. Unused.
**Legacy `PulsarClient` class:**
- `trustgraph-base/trustgraph/base/pubsub.py` — The `PulsarClient` class is a leftover from before the backend abstraction. `get_pubsub()` still references `PulsarClient.default_pulsar_host` for defaults. Move the defaults to `PulsarBackend` or to environment variable reads in the factory, then delete `PulsarClient`.
**Client libraries using Pulsar directly:**
- `trustgraph-base/trustgraph/clients/base.py` — `import pulsar`, `import _pulsar`, `from pulsar.schema import JsonSchema`. This is the base class for the old synchronous client library. These clients predate the backend abstraction and use Pulsar directly.
- `trustgraph-base/trustgraph/clients/embeddings_client.py` — `from pulsar.schema import JsonSchema`, `import _pulsar`.
- `trustgraph-base/trustgraph/clients/*.py` (agent, config, document_embeddings, document_rag, graph_embeddings, graph_rag, llm, prompt, row_embeddings, triples_query) — all import `_pulsar` for exception handling.
These clients are the internal request-response clients used by processors. They need to be migrated to use the backend abstraction or their Pulsar exception handling needs to be wrapped behind a backend-agnostic exception type.
**Translator base class:**
- `trustgraph-base/trustgraph/messaging/translators/base.py` — `from pulsar.schema import Record`. Used in type hints. Should be removed when `to_pulsar`/`from_pulsar` are renamed.
**Gateway service (bypasses abstraction):**
- `trustgraph-flow/trustgraph/gateway/service.py` — `import pulsar`. Creates a Pulsar client directly.
- `trustgraph-flow/trustgraph/gateway/config/receiver.py` — `import pulsar`. Direct Pulsar usage.
The gateway should use `get_pubsub()` like everything else.
**Storage writers:**
- `trustgraph-flow/trustgraph/storage/triples/neo4j/write.py` — `import pulsar`
- `trustgraph-flow/trustgraph/storage/triples/memgraph/write.py` — `import pulsar`
- `trustgraph-flow/trustgraph/storage/triples/falkordb/write.py` — `import pulsar`
- `trustgraph-flow/trustgraph/storage/triples/cassandra/write.py` — `import pulsar`
These need investigation — likely Pulsar exception handling or direct client usage that should go through the abstraction.
**Log level:**
- `trustgraph-base/trustgraph/log_level.py` — `import _pulsar`. Used to set Pulsar's log level. Should be moved into `pulsar_backend.py`.
### Queue naming
The current scheme encodes QoS, tenant, namespace, and queue name into a slash-separated string (`q0/tg/request/config`) which the Pulsar backend parses and maps to a Pulsar URI (`non-persistent://tg/request/config`). This was an attempt at abstraction but it has problems:
- QoS in the name was a mistake — it's a property of the queue definition, not something that belongs in the name. A queue is either persistent or it isn't; that's decided once when the queue is defined.
- The tenant/namespace structure mirrors Pulsar's model. RabbitMQ doesn't use this — it has vhosts and exchange/queue names. Pretending the naming isn't TrustGraph-specific just leaks Pulsar concepts.
- The `topic()` helper generates these strings, and the backend parses them apart. This is unnecessary indirection.
There are two categories of queue in TrustGraph:
**Infrastructure queues** — defined in code, used for system services. These are fixed and well-known:
| Queue | Persistent | Purpose |
|-------|------------|---------|
| `config-request` | No | Config queries |
| `config-response` | No | Config query responses |
| `config-push` | Yes | Config state broadcast |
| `flow-request` | No | Flow management queries |
| `flow-response` | No | Flow management responses |
| `librarian-request` | No | Document storage operations |
| `librarian-response` | No | Document storage responses |
| `knowledge-request` | No | Knowledge graph operations |
| `knowledge-response` | No | Knowledge graph responses |
| `document-embeddings-request` | No | Document vector queries |
| `document-embeddings-response` | No | Document vector responses |
| `row-embeddings-request` | No | Row vector queries |
| `row-embeddings-response` | No | Row vector responses |
| `collection-request` | No | Collection management |
| `collection-response` | No | Collection management responses |
**Flow queues** — defined in configuration, created dynamically per flow. The queue names come from the config service (e.g. `text-completion-request`, `graph-rag-request`, `agent-request`). Each flow instance has its own set of these queues.
For infrastructure queues, the name is just a string. Persistence is a property of the queue definition, not encoded in the name. The backend maps the name to whatever its native format requires.
For flow queues, the name comes from configuration. The config service already distributes queue names as strings — the backend just needs to be able to use them.
#### Proposed scheme: CLASS:TOPICSPACE:TOPIC
A queue name has three parts separated by colons:
- **CLASS** — a small enum that defines the queue's operational characteristics. The backend knows what each class means in terms of persistence, TTL, memory limits, etc. There are only four classes:
| Class | Persistent | TTL | Behaviour |
|-------|------------|-----|-----------|
| `flow` | Yes | Long | Processing pipeline queues. Messages survive broker restart. |
| `request` | No | Short | Transient request-response. Low TTL, no persistence needed — clients retry on failure. |
| `response` | No | Short | Same as request, for the response side. |
| `state` | Yes | Retained | Last-value state broadcast. Consumers need the most recent value on startup, plus any future updates. Config push is the primary example. |
- **TOPICSPACE** — deployment isolation. Keeps different TrustGraph deployments separate when sharing the same pub/sub infrastructure. Most deployments just use `tg`. Avoids the overloaded terms "tenant" and "namespace".
- **TOPIC** — the logical queue identity. What the queue is for.
**Examples:**
```
flow:tg:text-completion-request
flow:tg:graph-rag-request
flow:tg:agent-request
request:tg:librarian
response:tg:librarian
request:tg:config
response:tg:config
state:tg:config
request:tg:flow
response:tg:flow
```
**Backend mapping:**
Each backend parses the three parts and maps them to its native concepts:
- **Pulsar**: `flow:tg:text-completion-request` → `persistent://tg/flow/text-completion-request`. Class maps to persistent/non-persistent and namespace. State class uses persistent topic with earliest consumer positioning.
- **RabbitMQ**: Topicspace maps to vhost. Class determines queue durability and TTL policy. State class uses a last-value queue (via plugin) or a fanout exchange pattern where each consumer gets the retained state on connect.
- **Kafka**: `flow.tg.text-completion-request` as topic name. Class determines retention and compaction policy. State class maps to a compacted topic (last value per key).
**Why this works:**
- The class enum is small and stable — adding a new class is rare and deliberate
- Queue properties (persistence, TTL) are implied by class, not encoded in the name
- Dynamic registration works naturally — the config service publishes `flow:tg:text-completion-request` and the backend knows how to declare it from the `flow` class
- The colon separator is unambiguous, easy to split, doesn't conflict with URIs or path separators that backends use internally
- No pretence of being generic — this is a TrustGraph convention, and that's fine
### Serialization boundary
**Decision: the backend owns the wire format.**
The contract between the Consumer/Producer layer and the backend is dataclass objects in, dataclass objects out:
- `send()` accepts a dataclass instance and properties dict
- `receive()` returns a message whose `value()` is a dataclass instance
What happens on the wire is the backend's concern. The Pulsar backend uses JSON (via `dataclass_to_dict` / `dict_to_dataclass`). A RabbitMQ backend would likely also use JSON. A future backend could use Protobuf, MessagePack, or Avro if the broker benefits from it.
The serialization helpers stay inside the backend that uses them — they are not shared infrastructure. Each backend brings its own serialization strategy. The Consumer/Producer layer never thinks about wire format.
### Gateway service
**Decision: the gateway uses the backend abstraction like any other component.**
The gateway currently bridges WebSocket/REST to Pulsar directly, bypassing the abstraction layer. It translates incoming API JSON to Pulsar schema objects, sends them, receives responses as Pulsar schema objects, and translates back to API JSON. Since the wire format is JSON in both directions, this is effectively a no-op round trip through the schema machinery.
With the backend abstraction, the gateway follows the same pattern as every other component:
1. Incoming API JSON → translator `decode()` → dataclass
2. Dataclass → backend `send()` (backend handles wire format)
3. Backend `receive()` → dataclass
4. Dataclass → translator `encode()` → API JSON → WebSocket/REST client
This is architecturally simple — one code path, no special cases. The gateway depends on the schema dataclasses and the translator layer, which it already does. The overhead of deserialize-then-reserialize is negligible for the message sizes involved. And it keeps all options open — if a future backend uses a non-JSON wire format, the gateway still works without changes.
## Implementation Order
### Phase 1: Rename translators
Rename `to_pulsar()` → `decode()`, `from_pulsar()` → `encode()` across all translator classes and call sites. Remove `from pulsar.schema import Record` from the translator base class. Mechanical find-and-replace, no behavioural changes.
### Phase 2: Queue naming
Replace the `topic()` helper with the CLASS:TOPICSPACE:TOPIC scheme. Update all queue definitions in `schema/services/*.py` and `schema/knowledge/*.py`. Update `PulsarBackend.map_topic()` to parse the new format. Verify all existing functionality still works with Pulsar.
### Phase 3: Clean up Pulsar leaks
Work through the concrete cleanups list: remove dead imports, delete the legacy `PulsarClient` class, migrate the client libraries and gateway to use the backend abstraction. After this phase, `pulsar` imports exist only in `pulsar_backend.py`.
### Phase 4: RabbitMQ backend
Implement `rabbitmq_backend.py` against the existing `PubSubBackend` Protocol. Map queue classes to RabbitMQ concepts: `flow` → durable queues, `request`/`response` → non-durable queues with TTL, `state` → RabbitMQ streams. Add `rabbitmq` as a backend option in the factory. Test end-to-end with `PUBSUB_BACKEND=rabbitmq`.
Phases 1-3 are safe to do on main — they don't change behaviour, just clean up. Phase 4 is additive — it adds a new backend without touching the existing one.
### Config distribution on RabbitMQ
The `state` queue class needs "start from earliest" semantics — a newly started processor must receive the current configuration state.
RabbitMQ Streams (available since 3.9) solve this directly. Streams are persistent, append-only logs that support consumer offset positioning. The RabbitMQ backend maps the `state` class to a stream, and consumers attach with offset `first` to read from the beginning, or `last` to read the most recent entry plus future updates.
Since config pushes are full state snapshots (not deltas), a consumer only needs the most recent entry. The RabbitMQ backend can use `last` offset positioning for `state` class consumers, which delivers the last message in the stream followed by any new messages. This matches the current behaviour where processors read config on startup and then react to updates.

11
docs/tech-specs/query-time-explainability.md
View file

@ -63,7 +63,11 @@ Explainability events stream to client as the query executes:
3. Edges selected with reasoning → event emitted
4. Answer synthesized → event emitted
Client receives `explain_id` and `explain_collection` to fetch full details.
Client receives `explain_id`, `explain_graph`, and `explain_triples` inline
in each explain message. The triples contain the full provenance data for
that step — no follow-up graph query needed. The `explain_id` serves as
the root entity URI within the triples. Data is also written to the
knowledge graph for later audit/analysis.
## URI Structure
@ -144,7 +148,8 @@ class GraphRagResponse:
response: str = ""
end_of_stream: bool = False
explain_id: str | None = None
explain_collection: str | None = None
explain_graph: str | None = None
explain_triples: list[Triple] = field(default_factory=list)
message_type: str = "" # "chunk" or "explain"
end_of_session: bool = False
```
@ -154,7 +159,7 @@ class GraphRagResponse:
| message_type | Purpose |
|--------------|---------|
| `chunk` | Response text (streaming or final) |
| `explain` | Explainability event with IRI reference |
| `explain` | Explainability event with inline provenance triples |
### Session Lifecycle

268
docs/tech-specs/sparql-query.md Normal file
View file

@ -0,0 +1,268 @@
# SPARQL Query Service Technical Specification
## Overview
A pub/sub-hosted SPARQL query service that accepts SPARQL queries, decomposes
them into triple pattern lookups via the existing triples query pub/sub
interface, performs in-memory joins/filters/projections, and returns SPARQL
result bindings.
This makes the triple store queryable using a standard graph query language
without coupling to any specific backend (Neo4j, Cassandra, FalkorDB, etc.).
## Goals
- **SPARQL 1.1 support**: SELECT, ASK, CONSTRUCT, DESCRIBE queries
- **Backend-agnostic**: query via the pub/sub triples interface, not direct
database access
- **Standard service pattern**: FlowProcessor with ConsumerSpec/ProducerSpec,
using TriplesClientSpec to call the triples query service
- **Correct SPARQL semantics**: proper BGP evaluation, joins, OPTIONAL, UNION,
FILTER, BIND, aggregation, solution modifiers (ORDER BY, LIMIT, OFFSET,
DISTINCT)
## Background
The triples query service provides a single-pattern lookup: given optional
(s, p, o) values, return matching triples. This is the equivalent of one
triple pattern in a SPARQL Basic Graph Pattern.
To evaluate a full SPARQL query, we need to:
1. Parse the SPARQL string into an algebra tree
2. Walk the algebra tree, issuing triple pattern lookups for each BGP pattern
3. Join results across patterns (nested-loop or hash join)
4. Apply filters, optionals, unions, and aggregations in-memory
5. Project and return the requested variables
rdflib (already a dependency) provides a SPARQL 1.1 parser and algebra
compiler. We use rdflib to parse queries into algebra trees, then evaluate
the algebra ourselves using the triples query client as the data source.
## Technical Design
### Architecture
```
pub/sub
[Client] ──request──> [SPARQL Query Service] ──triples-request──> [Triples Query Service]
[Client] <─response── [SPARQL Query Service] <─triples-response── [Triples Query Service]
```
The service is a FlowProcessor that:
- Consumes SPARQL query requests
- Uses TriplesClientSpec to issue triple pattern lookups
- Evaluates the SPARQL algebra in-memory
- Produces result responses
### Components
1. **SPARQL Query Service (FlowProcessor)**
- ConsumerSpec for incoming SPARQL requests
- ProducerSpec for outgoing results
- TriplesClientSpec for calling the triples query service
- Delegates parsing and evaluation to the components below
Module: `trustgraph-flow/trustgraph/query/sparql/service.py`
2. **SPARQL Parser (rdflib wrapper)**
- Uses `rdflib.plugins.sparql.prepareQuery` / `parseQuery` and
`rdflib.plugins.sparql.algebra.translateQuery` to produce an algebra tree
- Extracts PREFIX declarations, query type (SELECT/ASK/CONSTRUCT/DESCRIBE),
and the algebra root
Module: `trustgraph-flow/trustgraph/query/sparql/parser.py`
3. **Algebra Evaluator**
- Recursive evaluator over the rdflib algebra tree
- Each algebra node type maps to an evaluation function
- BGP nodes issue triple pattern queries via TriplesClient
- Join/Filter/Optional/Union etc. operate on in-memory solution sequences
Module: `trustgraph-flow/trustgraph/query/sparql/algebra.py`
4. **Solution Sequence**
- A solution is a dict mapping variable names to Term values
- Solution sequences are lists of solutions
- Join: hash join on shared variables
- LeftJoin (OPTIONAL): hash join preserving unmatched left rows
- Union: concatenation
- Filter: evaluate SPARQL expressions against each solution
- Projection/Distinct/Order/Slice: standard post-processing
Module: `trustgraph-flow/trustgraph/query/sparql/solutions.py`
### Data Models
#### Request
```python
@dataclass
class SparqlQueryRequest:
user: str = ""
collection: str = ""
query: str = "" # SPARQL query string
limit: int = 10000 # Safety limit on results
```
#### Response
```python
@dataclass
class SparqlQueryResponse:
error: Error | None = None
query_type: str = "" # "select", "ask", "construct", "describe"
# For SELECT queries
variables: list[str] = field(default_factory=list)
bindings: list[SparqlBinding] = field(default_factory=list)
# For ASK queries
ask_result: bool = False
# For CONSTRUCT/DESCRIBE queries
triples: list[Triple] = field(default_factory=list)
@dataclass
class SparqlBinding:
values: list[Term | None] = field(default_factory=list)
```
### BGP Evaluation Strategy
For each triple pattern in a BGP:
- Extract bound terms (concrete IRIs/literals) and variables
- Call `TriplesClient.query_stream(s, p, o)` with bound terms, None for
variables
- Map returned triples back to variable bindings
For multi-pattern BGPs, join solutions incrementally:
- Order patterns by selectivity (patterns with more bound terms first)
- For each subsequent pattern, substitute bound variables from the current
solution sequence before querying
- This avoids full cross-products and reduces the number of triples queries
### Streaming and Early Termination
The triples query service supports streaming responses (batched delivery via
`TriplesClient.query_stream`). The SPARQL evaluator should use streaming
from the start, not as an optimisation. This is important because:
- **Early termination**: when the SPARQL query has a LIMIT, or when only one
solution is needed (ASK queries), we can stop consuming triples as soon as
we have enough results. Without streaming, a wildcard pattern like
`?s ?p ?o` would fetch the entire graph before we could apply the limit.
- **Memory efficiency**: results are processed batch-by-batch rather than
materialising the full result set in memory before joining.
The batch callback in `query_stream` returns a boolean to signal completion.
The evaluator should signal completion (return True) as soon as sufficient
solutions have been produced, allowing the underlying pub/sub consumer to
stop pulling batches.
### Parallel BGP Execution (Phase 2 Optimisation)
Within a BGP, patterns that share variables benefit from sequential
evaluation with bound-variable substitution (query results from earlier
patterns narrow later queries). However, patterns with no shared variables
are independent and could be issued concurrently via `asyncio.gather`.
A practical approach for a future optimisation pass:
- Analyse BGP patterns and identify connected components (groups of
patterns linked by shared variables)
- Execute independent components in parallel
- Within each component, evaluate patterns sequentially with substitution
This is not needed for correctness -- the sequential approach works for all
cases -- but could significantly reduce latency for queries with independent
pattern groups. Flagged as a phase 2 optimisation.
### FILTER Expression Evaluation
rdflib's algebra represents FILTER expressions as expression trees. We
evaluate these against each solution row, supporting:
- Comparison operators (=, !=, <, >, <=, >=)
- Logical operators (&&, ||, !)
- SPARQL built-in functions (isIRI, isLiteral, isBlank, str, lang,
datatype, bound, regex, etc.)
- Arithmetic operators (+, -, *, /)
## Implementation Order
1. **Schema and service skeleton** -- define SparqlQueryRequest/Response
dataclasses, create the FlowProcessor subclass with ConsumerSpec,
ProducerSpec, and TriplesClientSpec wired up. Verify it starts and
connects.
2. **SPARQL parsing** -- wrap rdflib's parser to produce algebra trees from
SPARQL strings. Handle parse errors gracefully. Unit test with a range of
query shapes.
3. **BGP evaluation** -- implement single-pattern and multi-pattern BGP
evaluation using TriplesClient. This is the core building block. Test
with simple SELECT WHERE { ?s ?p ?o } queries.
4. **Joins and solution sequences** -- implement hash join, left join (for
OPTIONAL), and union. Test with multi-pattern queries.
5. **FILTER evaluation** -- implement the expression evaluator for FILTER
clauses. Start with comparisons and logical operators, then add built-in
functions incrementally.
6. **Solution modifiers** -- DISTINCT, ORDER BY, LIMIT, OFFSET, projection.
7. **ASK / CONSTRUCT / DESCRIBE** -- extend beyond SELECT. ASK is trivial
(non-empty result = true). CONSTRUCT builds triples from a template.
DESCRIBE fetches all triples for matched resources.
8. **Aggregation** -- GROUP BY, HAVING, COUNT, SUM, AVG, MIN, MAX,
GROUP_CONCAT, SAMPLE.
9. **BIND, VALUES, subqueries** -- remaining SPARQL 1.1 features.
10. **API gateway integration** -- add SparqlQueryRequestor dispatcher,
request/response translators, and API endpoint so that the SPARQL
service is accessible via the HTTP gateway.
11. **SDK support** -- add `sparql_query()` method to FlowInstance in the
Python API SDK, following the same pattern as `triples_query()`.
12. **CLI command** -- add a `tg-sparql-query` CLI command that takes a
SPARQL query string (or reads from a file/stdin), submits it via the
SDK, and prints results in a readable format (table for SELECT,
true/false for ASK, Turtle for CONSTRUCT/DESCRIBE).
## Performance Considerations
In-memory join over pub/sub round-trips will be slower than native SPARQL on
a graph database. Key mitigations:
- **Streaming with early termination**: use `query_stream` so that
limit-bound queries don't fetch entire result sets. A `SELECT ... LIMIT 1`
against a wildcard pattern fetches one batch, not the whole graph.
- **Bound-variable substitution**: when evaluating BGP patterns sequentially,
substitute known bindings into subsequent patterns to issue narrow queries
rather than broad ones followed by in-memory filtering.
- **Parallel independent patterns** (phase 2): patterns with no shared
variables can be issued concurrently.
- **Query complexity limits**: may need a cap on the number of triple pattern
queries issued per SPARQL query to prevent runaway evaluation.
### Named Graph Mapping
SPARQL's `GRAPH ?g { ... }` and `GRAPH <uri> { ... }` clauses map to the
triples query service's graph filter parameter:
- `GRAPH <uri> { ?s ?p ?o }` — pass `g=uri` to the triples query
- Patterns outside any GRAPH clause — pass `g=""` (default graph only)
- `GRAPH ?g { ?s ?p ?o }` — pass `g="*"` (all graphs), then bind `?g` from
the returned triple's graph field
The triples query interface does not support a wildcard graph natively in
the SPARQL sense, but `g="*"` (all graphs) combined with client-side
filtering on the returned graph values achieves the same effect.
## Open Questions
- **SPARQL 1.2**: rdflib's parser support for 1.2 features (property paths
are already in 1.1; 1.2 adds lateral joins, ADJUST, etc.). Start with
1.1 and extend as rdflib support matures.

142
docs/websocket.html
View file

File diff suppressed because one or more lines are too long