Update tests for agent-orchestrator (#745)
Add 96 tests covering the orchestrator's aggregation, provenance,
routing, and explainability parsing. These verify the supervisor
fan-out/fan-in lifecycle, the new RDF provenance types
(Decomposition, Finding, Plan, StepResult, Synthesis), and their
round-trip through the wire format.
Unit tests (84):
- Aggregator: register, record completion, peek, build synthesis,
cleanup
- Provenance triple builders: types, provenance links,
goals/steps, labels
- Explainability parsing: from_triples dispatch, field extraction
for all new entity types, precedence over existing types
- PatternBase: is_subagent detection, emit_subagent_completion
message shape
- Completion dispatch: detection logic, full aggregator
integration flow, synthesis request not re-intercepted as
completion
- MetaRouter: task type identification, pattern selection,
valid_patterns constraints, fallback on LLM error or unknown
response
Contract tests (12):
- Orchestration fields on AgentRequest round-trip correctly
- subagent-completion and synthesise step types in request
history
- Plan steps with status and dependencies
- Provenance triple builder → wire format → from_triples
round-trip for all five new entity types
2026-03-31 13:12:26 +01:00
|
|
|
"""
|
|
|
|
|
Unit tests for explainability API parsing — verifies that from_triples()
|
|
|
|
|
correctly dispatches and parses the new orchestrator entity types.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
import pytest
|
|
|
|
|
|
|
|
|
|
from trustgraph.api.explainability import (
|
|
|
|
|
ExplainEntity,
|
|
|
|
|
Decomposition,
|
|
|
|
|
Finding,
|
|
|
|
|
Plan,
|
|
|
|
|
StepResult,
|
|
|
|
|
Synthesis,
|
|
|
|
|
Analysis,
|
Split Analysis into Analysis+ToolUse and Observation, add message_id (#747)
Refactor agent provenance so that the decision (thought + tool
selection) and the result (observation) are separate DAG entities:
Question ← Analysis+ToolUse ← Observation ← ... ← Conclusion
Analysis gains tg:ToolUse as a mixin RDF type and is emitted
before tool execution via an on_action callback in react().
This ensures sub-traces (e.g. GraphRAG) appear after their
parent Analysis in the streaming event order.
Observation becomes a standalone prov:Entity with tg:Observation
type, emitted after tool execution. The linear DAG chain runs
through Observation — subsequent iterations and the Conclusion
derive from it, not from the Analysis.
message_id is populated on streaming AgentResponse for thought
and observation chunks, using the provenance URI of the entity
being built. This lets clients group streamed chunks by entity.
Wire changes:
- provenance/agent.py: Add ToolUse type, new
agent_observation_triples(), remove observation from iteration
- agent_manager.py: Add on_action callback between reason() and
tool execution
- orchestrator/pattern_base.py: Split emit, wire message_id,
chain through observation URIs
- orchestrator/react_pattern.py: Emit Analysis via on_action
before tool runs
- agent/react/service.py: Same for non-orchestrator path
- api/explainability.py: New Observation class, updated dispatch
and chain walker
- api/types.py: Add message_id to AgentThought/AgentObservation
- cli: Render Observation separately, [analysis: tool] labels
2026-03-31 17:51:22 +01:00
|
|
|
Observation,
|
Update tests for agent-orchestrator (#745)
Add 96 tests covering the orchestrator's aggregation, provenance,
routing, and explainability parsing. These verify the supervisor
fan-out/fan-in lifecycle, the new RDF provenance types
(Decomposition, Finding, Plan, StepResult, Synthesis), and their
round-trip through the wire format.
Unit tests (84):
- Aggregator: register, record completion, peek, build synthesis,
cleanup
- Provenance triple builders: types, provenance links,
goals/steps, labels
- Explainability parsing: from_triples dispatch, field extraction
for all new entity types, precedence over existing types
- PatternBase: is_subagent detection, emit_subagent_completion
message shape
- Completion dispatch: detection logic, full aggregator
integration flow, synthesis request not re-intercepted as
completion
- MetaRouter: task type identification, pattern selection,
valid_patterns constraints, fallback on LLM error or unknown
response
Contract tests (12):
- Orchestration fields on AgentRequest round-trip correctly
- subagent-completion and synthesise step types in request
history
- Plan steps with status and dependencies
- Provenance triple builder → wire format → from_triples
round-trip for all five new entity types
2026-03-31 13:12:26 +01:00
|
|
|
Conclusion,
|
|
|
|
|
TG_DECOMPOSITION,
|
|
|
|
|
TG_FINDING,
|
|
|
|
|
TG_PLAN_TYPE,
|
|
|
|
|
TG_STEP_RESULT,
|
|
|
|
|
TG_SYNTHESIS,
|
|
|
|
|
TG_ANSWER_TYPE,
|
Split Analysis into Analysis+ToolUse and Observation, add message_id (#747)
Refactor agent provenance so that the decision (thought + tool
selection) and the result (observation) are separate DAG entities:
Question ← Analysis+ToolUse ← Observation ← ... ← Conclusion
Analysis gains tg:ToolUse as a mixin RDF type and is emitted
before tool execution via an on_action callback in react().
This ensures sub-traces (e.g. GraphRAG) appear after their
parent Analysis in the streaming event order.
Observation becomes a standalone prov:Entity with tg:Observation
type, emitted after tool execution. The linear DAG chain runs
through Observation — subsequent iterations and the Conclusion
derive from it, not from the Analysis.
message_id is populated on streaming AgentResponse for thought
and observation chunks, using the provenance URI of the entity
being built. This lets clients group streamed chunks by entity.
Wire changes:
- provenance/agent.py: Add ToolUse type, new
agent_observation_triples(), remove observation from iteration
- agent_manager.py: Add on_action callback between reason() and
tool execution
- orchestrator/pattern_base.py: Split emit, wire message_id,
chain through observation URIs
- orchestrator/react_pattern.py: Emit Analysis via on_action
before tool runs
- agent/react/service.py: Same for non-orchestrator path
- api/explainability.py: New Observation class, updated dispatch
and chain walker
- api/types.py: Add message_id to AgentThought/AgentObservation
- cli: Render Observation separately, [analysis: tool] labels
2026-03-31 17:51:22 +01:00
|
|
|
TG_OBSERVATION_TYPE,
|
Additional agent DAG tests (#750)
- test_agent_provenance.py: test_session_parent_uri,
test_session_no_parent_uri, and 6 synthesis tests (types,
single/multiple parents, document, label)
- test_on_action_callback.py: 3 tests — fires before tool, skipped
for Final, works when None
- test_callback_message_id.py: 7 tests — message_id on think/observe/
answer callbacks (streaming + non-streaming) and
send_final_response
- test_parse_chunk_message_id.py (5 tests) - _parse_chunk propagates
message_id for thought, observation, answer; handles missing
gracefully
- test_explainability_parsing.py (+1) -
test_dispatches_analysis_with_tooluse - Analysis+ToolUse mixin still
dispatches to Analysis
- test_explainability.py (+1) -
test_observation_found_via_subtrace_synthesis
- chain walker follows from sub-trace Synthesis to find Observation
and
Conclusion in correct order
- test_agent_provenance.py (+8) - session parent_uri (2), synthesis
single/multiple parents, types, document, label (6)
2026-04-01 13:59:34 +01:00
|
|
|
TG_TOOL_USE,
|
Update tests for agent-orchestrator (#745)
Add 96 tests covering the orchestrator's aggregation, provenance,
routing, and explainability parsing. These verify the supervisor
fan-out/fan-in lifecycle, the new RDF provenance types
(Decomposition, Finding, Plan, StepResult, Synthesis), and their
round-trip through the wire format.
Unit tests (84):
- Aggregator: register, record completion, peek, build synthesis,
cleanup
- Provenance triple builders: types, provenance links,
goals/steps, labels
- Explainability parsing: from_triples dispatch, field extraction
for all new entity types, precedence over existing types
- PatternBase: is_subagent detection, emit_subagent_completion
message shape
- Completion dispatch: detection logic, full aggregator
integration flow, synthesis request not re-intercepted as
completion
- MetaRouter: task type identification, pattern selection,
valid_patterns constraints, fallback on LLM error or unknown
response
Contract tests (12):
- Orchestration fields on AgentRequest round-trip correctly
- subagent-completion and synthesise step types in request
history
- Plan steps with status and dependencies
- Provenance triple builder → wire format → from_triples
round-trip for all five new entity types
2026-03-31 13:12:26 +01:00
|
|
|
TG_ANALYSIS,
|
|
|
|
|
TG_CONCLUSION,
|
|
|
|
|
TG_DOCUMENT,
|
|
|
|
|
TG_SUBAGENT_GOAL,
|
|
|
|
|
TG_PLAN_STEP,
|
|
|
|
|
RDF_TYPE,
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
PROV_ENTITY = "http://www.w3.org/ns/prov#Entity"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _make_triples(uri, types, extras=None):
|
|
|
|
|
"""Build a list of (s, p, o) tuples for testing."""
|
|
|
|
|
triples = [(uri, RDF_TYPE, t) for t in types]
|
|
|
|
|
if extras:
|
|
|
|
|
triples.extend((uri, p, o) for p, o in extras)
|
|
|
|
|
return triples
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class TestFromTriplesDispatch:
|
|
|
|
|
|
|
|
|
|
def test_dispatches_decomposition(self):
|
|
|
|
|
triples = _make_triples("urn:d", [PROV_ENTITY, TG_DECOMPOSITION])
|
|
|
|
|
entity = ExplainEntity.from_triples("urn:d", triples)
|
|
|
|
|
assert isinstance(entity, Decomposition)
|
|
|
|
|
|
|
|
|
|
def test_dispatches_finding(self):
|
|
|
|
|
triples = _make_triples("urn:f",
|
|
|
|
|
[PROV_ENTITY, TG_FINDING, TG_ANSWER_TYPE])
|
|
|
|
|
entity = ExplainEntity.from_triples("urn:f", triples)
|
|
|
|
|
assert isinstance(entity, Finding)
|
|
|
|
|
|
|
|
|
|
def test_dispatches_plan(self):
|
|
|
|
|
triples = _make_triples("urn:p", [PROV_ENTITY, TG_PLAN_TYPE])
|
|
|
|
|
entity = ExplainEntity.from_triples("urn:p", triples)
|
|
|
|
|
assert isinstance(entity, Plan)
|
|
|
|
|
|
|
|
|
|
def test_dispatches_step_result(self):
|
|
|
|
|
triples = _make_triples("urn:sr",
|
|
|
|
|
[PROV_ENTITY, TG_STEP_RESULT, TG_ANSWER_TYPE])
|
|
|
|
|
entity = ExplainEntity.from_triples("urn:sr", triples)
|
|
|
|
|
assert isinstance(entity, StepResult)
|
|
|
|
|
|
|
|
|
|
def test_dispatches_synthesis(self):
|
|
|
|
|
triples = _make_triples("urn:s",
|
|
|
|
|
[PROV_ENTITY, TG_SYNTHESIS, TG_ANSWER_TYPE])
|
|
|
|
|
entity = ExplainEntity.from_triples("urn:s", triples)
|
|
|
|
|
assert isinstance(entity, Synthesis)
|
|
|
|
|
|
|
|
|
|
def test_dispatches_analysis_unchanged(self):
|
|
|
|
|
triples = _make_triples("urn:a", [PROV_ENTITY, TG_ANALYSIS])
|
|
|
|
|
entity = ExplainEntity.from_triples("urn:a", triples)
|
|
|
|
|
assert isinstance(entity, Analysis)
|
|
|
|
|
|
Additional agent DAG tests (#750)
- test_agent_provenance.py: test_session_parent_uri,
test_session_no_parent_uri, and 6 synthesis tests (types,
single/multiple parents, document, label)
- test_on_action_callback.py: 3 tests — fires before tool, skipped
for Final, works when None
- test_callback_message_id.py: 7 tests — message_id on think/observe/
answer callbacks (streaming + non-streaming) and
send_final_response
- test_parse_chunk_message_id.py (5 tests) - _parse_chunk propagates
message_id for thought, observation, answer; handles missing
gracefully
- test_explainability_parsing.py (+1) -
test_dispatches_analysis_with_tooluse - Analysis+ToolUse mixin still
dispatches to Analysis
- test_explainability.py (+1) -
test_observation_found_via_subtrace_synthesis
- chain walker follows from sub-trace Synthesis to find Observation
and
Conclusion in correct order
- test_agent_provenance.py (+8) - session parent_uri (2), synthesis
single/multiple parents, types, document, label (6)
2026-04-01 13:59:34 +01:00
|
|
|
def test_dispatches_analysis_with_tooluse(self):
|
|
|
|
|
"""Analysis+ToolUse mixin still dispatches to Analysis."""
|
|
|
|
|
triples = _make_triples("urn:a",
|
|
|
|
|
[PROV_ENTITY, TG_ANALYSIS, TG_TOOL_USE])
|
|
|
|
|
entity = ExplainEntity.from_triples("urn:a", triples)
|
|
|
|
|
assert isinstance(entity, Analysis)
|
|
|
|
|
|
Split Analysis into Analysis+ToolUse and Observation, add message_id (#747)
Refactor agent provenance so that the decision (thought + tool
selection) and the result (observation) are separate DAG entities:
Question ← Analysis+ToolUse ← Observation ← ... ← Conclusion
Analysis gains tg:ToolUse as a mixin RDF type and is emitted
before tool execution via an on_action callback in react().
This ensures sub-traces (e.g. GraphRAG) appear after their
parent Analysis in the streaming event order.
Observation becomes a standalone prov:Entity with tg:Observation
type, emitted after tool execution. The linear DAG chain runs
through Observation — subsequent iterations and the Conclusion
derive from it, not from the Analysis.
message_id is populated on streaming AgentResponse for thought
and observation chunks, using the provenance URI of the entity
being built. This lets clients group streamed chunks by entity.
Wire changes:
- provenance/agent.py: Add ToolUse type, new
agent_observation_triples(), remove observation from iteration
- agent_manager.py: Add on_action callback between reason() and
tool execution
- orchestrator/pattern_base.py: Split emit, wire message_id,
chain through observation URIs
- orchestrator/react_pattern.py: Emit Analysis via on_action
before tool runs
- agent/react/service.py: Same for non-orchestrator path
- api/explainability.py: New Observation class, updated dispatch
and chain walker
- api/types.py: Add message_id to AgentThought/AgentObservation
- cli: Render Observation separately, [analysis: tool] labels
2026-03-31 17:51:22 +01:00
|
|
|
def test_dispatches_observation(self):
|
|
|
|
|
triples = _make_triples("urn:o", [PROV_ENTITY, TG_OBSERVATION_TYPE])
|
|
|
|
|
entity = ExplainEntity.from_triples("urn:o", triples)
|
|
|
|
|
assert isinstance(entity, Observation)
|
|
|
|
|
|
Update tests for agent-orchestrator (#745)
Add 96 tests covering the orchestrator's aggregation, provenance,
routing, and explainability parsing. These verify the supervisor
fan-out/fan-in lifecycle, the new RDF provenance types
(Decomposition, Finding, Plan, StepResult, Synthesis), and their
round-trip through the wire format.
Unit tests (84):
- Aggregator: register, record completion, peek, build synthesis,
cleanup
- Provenance triple builders: types, provenance links,
goals/steps, labels
- Explainability parsing: from_triples dispatch, field extraction
for all new entity types, precedence over existing types
- PatternBase: is_subagent detection, emit_subagent_completion
message shape
- Completion dispatch: detection logic, full aggregator
integration flow, synthesis request not re-intercepted as
completion
- MetaRouter: task type identification, pattern selection,
valid_patterns constraints, fallback on LLM error or unknown
response
Contract tests (12):
- Orchestration fields on AgentRequest round-trip correctly
- subagent-completion and synthesise step types in request
history
- Plan steps with status and dependencies
- Provenance triple builder → wire format → from_triples
round-trip for all five new entity types
2026-03-31 13:12:26 +01:00
|
|
|
def test_dispatches_conclusion_unchanged(self):
|
|
|
|
|
triples = _make_triples("urn:c",
|
|
|
|
|
[PROV_ENTITY, TG_CONCLUSION, TG_ANSWER_TYPE])
|
|
|
|
|
entity = ExplainEntity.from_triples("urn:c", triples)
|
|
|
|
|
assert isinstance(entity, Conclusion)
|
|
|
|
|
|
|
|
|
|
def test_finding_takes_precedence_over_synthesis(self):
|
|
|
|
|
"""Finding has Answer mixin but should dispatch to Finding, not
|
|
|
|
|
Synthesis, because Finding is checked first."""
|
|
|
|
|
triples = _make_triples("urn:f",
|
|
|
|
|
[PROV_ENTITY, TG_FINDING, TG_ANSWER_TYPE])
|
|
|
|
|
entity = ExplainEntity.from_triples("urn:f", triples)
|
|
|
|
|
assert isinstance(entity, Finding)
|
|
|
|
|
assert not isinstance(entity, Synthesis)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class TestDecompositionParsing:
|
|
|
|
|
|
|
|
|
|
def test_parses_goals(self):
|
|
|
|
|
triples = _make_triples("urn:d", [TG_DECOMPOSITION], [
|
|
|
|
|
(TG_SUBAGENT_GOAL, "What is X?"),
|
|
|
|
|
(TG_SUBAGENT_GOAL, "What is Y?"),
|
|
|
|
|
])
|
|
|
|
|
entity = Decomposition.from_triples("urn:d", triples)
|
|
|
|
|
assert set(entity.goals) == {"What is X?", "What is Y?"}
|
|
|
|
|
|
|
|
|
|
def test_entity_type_field(self):
|
|
|
|
|
triples = _make_triples("urn:d", [TG_DECOMPOSITION])
|
|
|
|
|
entity = Decomposition.from_triples("urn:d", triples)
|
|
|
|
|
assert entity.entity_type == "decomposition"
|
|
|
|
|
|
|
|
|
|
def test_empty_goals(self):
|
|
|
|
|
triples = _make_triples("urn:d", [TG_DECOMPOSITION])
|
|
|
|
|
entity = Decomposition.from_triples("urn:d", triples)
|
|
|
|
|
assert entity.goals == []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class TestFindingParsing:
|
|
|
|
|
|
|
|
|
|
def test_parses_goal_and_document(self):
|
|
|
|
|
triples = _make_triples("urn:f", [TG_FINDING, TG_ANSWER_TYPE], [
|
|
|
|
|
(TG_SUBAGENT_GOAL, "What is X?"),
|
|
|
|
|
(TG_DOCUMENT, "urn:doc/finding"),
|
|
|
|
|
])
|
|
|
|
|
entity = Finding.from_triples("urn:f", triples)
|
|
|
|
|
assert entity.goal == "What is X?"
|
|
|
|
|
assert entity.document == "urn:doc/finding"
|
|
|
|
|
|
|
|
|
|
def test_entity_type_field(self):
|
|
|
|
|
triples = _make_triples("urn:f", [TG_FINDING])
|
|
|
|
|
entity = Finding.from_triples("urn:f", triples)
|
|
|
|
|
assert entity.entity_type == "finding"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class TestPlanParsing:
|
|
|
|
|
|
|
|
|
|
def test_parses_steps(self):
|
|
|
|
|
triples = _make_triples("urn:p", [TG_PLAN_TYPE], [
|
|
|
|
|
(TG_PLAN_STEP, "Define X"),
|
|
|
|
|
(TG_PLAN_STEP, "Research Y"),
|
|
|
|
|
(TG_PLAN_STEP, "Analyse Z"),
|
|
|
|
|
])
|
|
|
|
|
entity = Plan.from_triples("urn:p", triples)
|
|
|
|
|
assert set(entity.steps) == {"Define X", "Research Y", "Analyse Z"}
|
|
|
|
|
|
|
|
|
|
def test_entity_type_field(self):
|
|
|
|
|
triples = _make_triples("urn:p", [TG_PLAN_TYPE])
|
|
|
|
|
entity = Plan.from_triples("urn:p", triples)
|
|
|
|
|
assert entity.entity_type == "plan"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class TestStepResultParsing:
|
|
|
|
|
|
|
|
|
|
def test_parses_step_and_document(self):
|
|
|
|
|
triples = _make_triples("urn:sr", [TG_STEP_RESULT, TG_ANSWER_TYPE], [
|
|
|
|
|
(TG_PLAN_STEP, "Define X"),
|
|
|
|
|
(TG_DOCUMENT, "urn:doc/step"),
|
|
|
|
|
])
|
|
|
|
|
entity = StepResult.from_triples("urn:sr", triples)
|
|
|
|
|
assert entity.step == "Define X"
|
|
|
|
|
assert entity.document == "urn:doc/step"
|
|
|
|
|
|
|
|
|
|
def test_entity_type_field(self):
|
|
|
|
|
triples = _make_triples("urn:sr", [TG_STEP_RESULT])
|
|
|
|
|
entity = StepResult.from_triples("urn:sr", triples)
|
|
|
|
|
assert entity.entity_type == "step-result"
|