apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-04-26 00:46:22 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Release/v1.2 (#457)

Browse source 
* Bump setup.py versions for 1.1

* PoC MCP server (#419)

* Very initial MCP server PoC for TrustGraph

* Put service on port 8000

* Add MCP container and packages to buildout

* Update docs for API/CLI changes in 1.0 (#421)

* Update some API basics for the 0.23/1.0 API change

* Add MCP container push (#425)

* Add command args to the MCP server (#426)

* Host and port parameters

* Added websocket arg

* More docs

* MCP client support (#427)

- MCP client service
- Tool request/response schema
- API gateway support for mcp-tool
- Message translation for tool request & response
- Make mcp-tool using configuration service for information
  about where the MCP services are.

* Feature/react call mcp (#428)

Key Features

  - MCP Tool Integration: Added core MCP tool support with ToolClientSpec and ToolClient classes
  - API Enhancement: New mcp_tool method for flow-specific tool invocation
  - CLI Tooling: New tg-invoke-mcp-tool command for testing MCP integration
  - React Agent Enhancement: Fixed and improved multi-tool invocation capabilities
  - Tool Management: Enhanced CLI for tool configuration and management

Changes

  - Added MCP tool invocation to API with flow-specific integration
  - Implemented ToolClientSpec and ToolClient for tool call handling
  - Updated agent-manager-react to invoke MCP tools with configurable types
  - Enhanced CLI with new commands and improved help text
  - Added comprehensive documentation for new CLI commands
  - Improved tool configuration management

Testing

  - Added tg-invoke-mcp-tool CLI command for isolated MCP integration testing
  - Enhanced agent capability to invoke multiple tools simultaneously

* Test suite executed from CI pipeline (#433)

* Test strategy & test cases

* Unit tests

* Integration tests

* Extending test coverage (#434)

* Contract tests

* Testing embeedings

* Agent unit tests

* Knowledge pipeline tests

* Turn on contract tests

* Increase storage test coverage (#435)

* Fixing storage and adding tests

* PR pipeline only runs quick tests

* Empty configuration is returned as empty list, previously was not in response (#436)

* Update config util to take files as well as command-line text (#437)

* Updated CLI invocation and config model for tools and mcp (#438)

* Updated CLI invocation and config model for tools and mcp

* CLI anomalies

* Tweaked the MCP tool implementation for new model

* Update agent implementation to match the new model

* Fix agent tools, now all tested

* Fixed integration tests

* Fix MCP delete tool params

* Update Python deps to 1.2

* Update to enable knowledge extraction using the agent framework (#439)

* Implement KG extraction agent (kg-extract-agent)

* Using ReAct framework (agent-manager-react)
 
* ReAct manager had an issue when emitting JSON, which conflicts which ReAct manager's own JSON messages, so refactored ReAct manager to use traditional ReAct messages, non-JSON structure.
 
* Minor refactor to take the prompt template client out of prompt-template so it can be more readily used by other modules. kg-extract-agent uses this framework.

* Migrate from setup.py to pyproject.toml (#440)

* Converted setup.py to pyproject.toml

* Modern package infrastructure as recommended by py docs

* Install missing build deps (#441)

* Install missing build deps (#442)

* Implement logging strategy (#444)

* Logging strategy and convert all prints() to logging invocations

* Fix/startup failure (#445)

* Fix loggin startup problems

* Fix logging startup problems (#446)

* Fix logging startup problems (#447)

* Fixed Mistral OCR to use current API (#448)

* Fixed Mistral OCR to use current API

* Added PDF decoder tests

* Fix Mistral OCR ident to be standard pdf-decoder (#450)

* Fix Mistral OCR ident to be standard pdf-decoder

* Correct test

* Schema structure refactor (#451)

* Write schema refactor spec

* Implemented schema refactor spec

* Structure data mvp (#452)

* Structured data tech spec

* Architecture principles

* New schemas

* Updated schemas and specs

* Object extractor

* Add .coveragerc

* New tests

* Cassandra object storage

* Trying to object extraction working, issues exist

* Validate librarian collection (#453)

* Fix token chunker, broken API invocation (#454)

* Fix token chunker, broken API invocation (#455)

* Knowledge load utility CLI (#456)

* Knowledge loader

* More tests
This commit is contained in:
cybermaggedon 2025-08-18 20:56:09 +01:00 • committed by GitHub
parent c85ba197be
commit 89be656990
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
509 changed files with 49632 additions and 5159 deletions
Download patch file Download diff file Expand all files Collapse all files

558
tests/unit/test_gateway/test_dispatch_manager.py Normal file
View file

@ -0,0 +1,558 @@
"""
Tests for Gateway Dispatcher Manager
"""
import pytest
import asyncio
from unittest.mock import Mock, patch, AsyncMock, MagicMock
import uuid
from trustgraph.gateway.dispatch.manager import DispatcherManager, DispatcherWrapper
# Keep the real methods intact for proper testing
class TestDispatcherWrapper:
"""Test cases for DispatcherWrapper class"""
def test_dispatcher_wrapper_initialization(self):
"""Test DispatcherWrapper initialization"""
mock_handler = Mock()
wrapper = DispatcherWrapper(mock_handler)
assert wrapper.handler == mock_handler
@pytest.mark.asyncio
async def test_dispatcher_wrapper_process(self):
"""Test DispatcherWrapper process method"""
mock_handler = AsyncMock()
wrapper = DispatcherWrapper(mock_handler)
result = await wrapper.process("arg1", "arg2")
mock_handler.assert_called_once_with("arg1", "arg2")
assert result == mock_handler.return_value
class TestDispatcherManager:
"""Test cases for DispatcherManager class"""
def test_dispatcher_manager_initialization(self):
"""Test DispatcherManager initialization"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
assert manager.pulsar_client == mock_pulsar_client
assert manager.config_receiver == mock_config_receiver
assert manager.prefix == "api-gateway" # default prefix
assert manager.flows == {}
assert manager.dispatchers == {}
# Verify manager was added as handler to config receiver
mock_config_receiver.add_handler.assert_called_once_with(manager)
def test_dispatcher_manager_initialization_with_custom_prefix(self):
"""Test DispatcherManager initialization with custom prefix"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver, prefix="custom-prefix")
assert manager.prefix == "custom-prefix"
@pytest.mark.asyncio
async def test_start_flow(self):
"""Test start_flow method"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
flow_data = {"name": "test_flow", "steps": []}
await manager.start_flow("flow1", flow_data)
assert "flow1" in manager.flows
assert manager.flows["flow1"] == flow_data
@pytest.mark.asyncio
async def test_stop_flow(self):
"""Test stop_flow method"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
# Pre-populate with a flow
flow_data = {"name": "test_flow", "steps": []}
manager.flows["flow1"] = flow_data
await manager.stop_flow("flow1", flow_data)
assert "flow1" not in manager.flows
def test_dispatch_global_service_returns_wrapper(self):
"""Test dispatch_global_service returns DispatcherWrapper"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
wrapper = manager.dispatch_global_service()
assert isinstance(wrapper, DispatcherWrapper)
assert wrapper.handler == manager.process_global_service
def test_dispatch_core_export_returns_wrapper(self):
"""Test dispatch_core_export returns DispatcherWrapper"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
wrapper = manager.dispatch_core_export()
assert isinstance(wrapper, DispatcherWrapper)
assert wrapper.handler == manager.process_core_export
def test_dispatch_core_import_returns_wrapper(self):
"""Test dispatch_core_import returns DispatcherWrapper"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
wrapper = manager.dispatch_core_import()
assert isinstance(wrapper, DispatcherWrapper)
assert wrapper.handler == manager.process_core_import
@pytest.mark.asyncio
async def test_process_core_import(self):
"""Test process_core_import method"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
with patch('trustgraph.gateway.dispatch.manager.CoreImport') as mock_core_import:
mock_importer = Mock()
mock_importer.process = AsyncMock(return_value="import_result")
mock_core_import.return_value = mock_importer
result = await manager.process_core_import("data", "error", "ok", "request")
mock_core_import.assert_called_once_with(mock_pulsar_client)
mock_importer.process.assert_called_once_with("data", "error", "ok", "request")
assert result == "import_result"
@pytest.mark.asyncio
async def test_process_core_export(self):
"""Test process_core_export method"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
with patch('trustgraph.gateway.dispatch.manager.CoreExport') as mock_core_export:
mock_exporter = Mock()
mock_exporter.process = AsyncMock(return_value="export_result")
mock_core_export.return_value = mock_exporter
result = await manager.process_core_export("data", "error", "ok", "request")
mock_core_export.assert_called_once_with(mock_pulsar_client)
mock_exporter.process.assert_called_once_with("data", "error", "ok", "request")
assert result == "export_result"
@pytest.mark.asyncio
async def test_process_global_service(self):
"""Test process_global_service method"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
manager.invoke_global_service = AsyncMock(return_value="global_result")
params = {"kind": "test_kind"}
result = await manager.process_global_service("data", "responder", params)
manager.invoke_global_service.assert_called_once_with("data", "responder", "test_kind")
assert result == "global_result"
@pytest.mark.asyncio
async def test_invoke_global_service_with_existing_dispatcher(self):
"""Test invoke_global_service with existing dispatcher"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
# Pre-populate with existing dispatcher
mock_dispatcher = Mock()
mock_dispatcher.process = AsyncMock(return_value="cached_result")
manager.dispatchers[(None, "config")] = mock_dispatcher
result = await manager.invoke_global_service("data", "responder", "config")
mock_dispatcher.process.assert_called_once_with("data", "responder")
assert result == "cached_result"
@pytest.mark.asyncio
async def test_invoke_global_service_creates_new_dispatcher(self):
"""Test invoke_global_service creates new dispatcher"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
with patch('trustgraph.gateway.dispatch.manager.global_dispatchers') as mock_dispatchers:
mock_dispatcher_class = Mock()
mock_dispatcher = Mock()
mock_dispatcher.start = AsyncMock()
mock_dispatcher.process = AsyncMock(return_value="new_result")
mock_dispatcher_class.return_value = mock_dispatcher
mock_dispatchers.__getitem__.return_value = mock_dispatcher_class
result = await manager.invoke_global_service("data", "responder", "config")
# Verify dispatcher was created with correct parameters
mock_dispatcher_class.assert_called_once_with(
pulsar_client=mock_pulsar_client,
timeout=120,
consumer="api-gateway-config-request",
subscriber="api-gateway-config-request"
)
mock_dispatcher.start.assert_called_once()
mock_dispatcher.process.assert_called_once_with("data", "responder")
# Verify dispatcher was cached
assert manager.dispatchers[(None, "config")] == mock_dispatcher
assert result == "new_result"
def test_dispatch_flow_import_returns_method(self):
"""Test dispatch_flow_import returns correct method"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
result = manager.dispatch_flow_import()
assert result == manager.process_flow_import
def test_dispatch_flow_export_returns_method(self):
"""Test dispatch_flow_export returns correct method"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
result = manager.dispatch_flow_export()
assert result == manager.process_flow_export
def test_dispatch_socket_returns_method(self):
"""Test dispatch_socket returns correct method"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
result = manager.dispatch_socket()
assert result == manager.process_socket
def test_dispatch_flow_service_returns_wrapper(self):
"""Test dispatch_flow_service returns DispatcherWrapper"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
wrapper = manager.dispatch_flow_service()
assert isinstance(wrapper, DispatcherWrapper)
assert wrapper.handler == manager.process_flow_service
@pytest.mark.asyncio
async def test_process_flow_import_with_valid_flow_and_kind(self):
"""Test process_flow_import with valid flow and kind"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
# Setup test flow
manager.flows["test_flow"] = {
"interfaces": {
"triples-store": {"queue": "test_queue"}
}
}
with patch('trustgraph.gateway.dispatch.manager.import_dispatchers') as mock_dispatchers, \
patch('uuid.uuid4') as mock_uuid:
mock_uuid.return_value = "test-uuid"
mock_dispatcher_class = Mock()
mock_dispatcher = Mock()
mock_dispatcher.start = AsyncMock()
mock_dispatcher_class.return_value = mock_dispatcher
mock_dispatchers.__getitem__.return_value = mock_dispatcher_class
mock_dispatchers.__contains__.return_value = True
params = {"flow": "test_flow", "kind": "triples"}
result = await manager.process_flow_import("ws", "running", params)
mock_dispatcher_class.assert_called_once_with(
pulsar_client=mock_pulsar_client,
ws="ws",
running="running",
queue={"queue": "test_queue"}
)
mock_dispatcher.start.assert_called_once()
assert result == mock_dispatcher
@pytest.mark.asyncio
async def test_process_flow_import_with_invalid_flow(self):
"""Test process_flow_import with invalid flow"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
params = {"flow": "invalid_flow", "kind": "triples"}
with pytest.raises(RuntimeError, match="Invalid flow"):
await manager.process_flow_import("ws", "running", params)
@pytest.mark.asyncio
async def test_process_flow_import_with_invalid_kind(self):
"""Test process_flow_import with invalid kind"""
import warnings
with warnings.catch_warnings():
warnings.simplefilter("ignore", RuntimeWarning)
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
# Setup test flow
manager.flows["test_flow"] = {
"interfaces": {
"triples-store": {"queue": "test_queue"}
}
}
with patch('trustgraph.gateway.dispatch.manager.import_dispatchers') as mock_dispatchers:
mock_dispatchers.__contains__.return_value = False
params = {"flow": "test_flow", "kind": "invalid_kind"}
with pytest.raises(RuntimeError, match="Invalid kind"):
await manager.process_flow_import("ws", "running", params)
@pytest.mark.asyncio
async def test_process_flow_export_with_valid_flow_and_kind(self):
"""Test process_flow_export with valid flow and kind"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
# Setup test flow
manager.flows["test_flow"] = {
"interfaces": {
"triples-store": {"queue": "test_queue"}
}
}
with patch('trustgraph.gateway.dispatch.manager.export_dispatchers') as mock_dispatchers, \
patch('uuid.uuid4') as mock_uuid:
mock_uuid.return_value = "test-uuid"
mock_dispatcher_class = Mock()
mock_dispatcher = Mock()
mock_dispatcher_class.return_value = mock_dispatcher
mock_dispatchers.__getitem__.return_value = mock_dispatcher_class
mock_dispatchers.__contains__.return_value = True
params = {"flow": "test_flow", "kind": "triples"}
result = await manager.process_flow_export("ws", "running", params)
mock_dispatcher_class.assert_called_once_with(
pulsar_client=mock_pulsar_client,
ws="ws",
running="running",
queue={"queue": "test_queue"},
consumer="api-gateway-test-uuid",
subscriber="api-gateway-test-uuid"
)
assert result == mock_dispatcher
@pytest.mark.asyncio
async def test_process_socket(self):
"""Test process_socket method"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
with patch('trustgraph.gateway.dispatch.manager.Mux') as mock_mux:
mock_mux_instance = Mock()
mock_mux.return_value = mock_mux_instance
result = await manager.process_socket("ws", "running", {})
mock_mux.assert_called_once_with(manager, "ws", "running")
assert result == mock_mux_instance
@pytest.mark.asyncio
async def test_process_flow_service(self):
"""Test process_flow_service method"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
manager.invoke_flow_service = AsyncMock(return_value="flow_result")
params = {"flow": "test_flow", "kind": "agent"}
result = await manager.process_flow_service("data", "responder", params)
manager.invoke_flow_service.assert_called_once_with("data", "responder", "test_flow", "agent")
assert result == "flow_result"
@pytest.mark.asyncio
async def test_invoke_flow_service_with_existing_dispatcher(self):
"""Test invoke_flow_service with existing dispatcher"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
# Add flow to the flows dictionary
manager.flows["test_flow"] = {"services": {"agent": {}}}
# Pre-populate with existing dispatcher
mock_dispatcher = Mock()
mock_dispatcher.process = AsyncMock(return_value="cached_result")
manager.dispatchers[("test_flow", "agent")] = mock_dispatcher
result = await manager.invoke_flow_service("data", "responder", "test_flow", "agent")
mock_dispatcher.process.assert_called_once_with("data", "responder")
assert result == "cached_result"
@pytest.mark.asyncio
async def test_invoke_flow_service_creates_request_response_dispatcher(self):
"""Test invoke_flow_service creates request-response dispatcher"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
# Setup test flow
manager.flows["test_flow"] = {
"interfaces": {
"agent": {
"request": "agent_request_queue",
"response": "agent_response_queue"
}
}
}
with patch('trustgraph.gateway.dispatch.manager.request_response_dispatchers') as mock_dispatchers:
mock_dispatcher_class = Mock()
mock_dispatcher = Mock()
mock_dispatcher.start = AsyncMock()
mock_dispatcher.process = AsyncMock(return_value="new_result")
mock_dispatcher_class.return_value = mock_dispatcher
mock_dispatchers.__getitem__.return_value = mock_dispatcher_class
mock_dispatchers.__contains__.return_value = True
result = await manager.invoke_flow_service("data", "responder", "test_flow", "agent")
# Verify dispatcher was created with correct parameters
mock_dispatcher_class.assert_called_once_with(
pulsar_client=mock_pulsar_client,
request_queue="agent_request_queue",
response_queue="agent_response_queue",
timeout=120,
consumer="api-gateway-test_flow-agent-request",
subscriber="api-gateway-test_flow-agent-request"
)
mock_dispatcher.start.assert_called_once()
mock_dispatcher.process.assert_called_once_with("data", "responder")
# Verify dispatcher was cached
assert manager.dispatchers[("test_flow", "agent")] == mock_dispatcher
assert result == "new_result"
@pytest.mark.asyncio
async def test_invoke_flow_service_creates_sender_dispatcher(self):
"""Test invoke_flow_service creates sender dispatcher"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
# Setup test flow
manager.flows["test_flow"] = {
"interfaces": {
"text-load": {"queue": "text_load_queue"}
}
}
with patch('trustgraph.gateway.dispatch.manager.request_response_dispatchers') as mock_rr_dispatchers, \
patch('trustgraph.gateway.dispatch.manager.sender_dispatchers') as mock_sender_dispatchers:
mock_rr_dispatchers.__contains__.return_value = False
mock_sender_dispatchers.__contains__.return_value = True
mock_dispatcher_class = Mock()
mock_dispatcher = Mock()
mock_dispatcher.start = AsyncMock()
mock_dispatcher.process = AsyncMock(return_value="sender_result")
mock_dispatcher_class.return_value = mock_dispatcher
mock_sender_dispatchers.__getitem__.return_value = mock_dispatcher_class
result = await manager.invoke_flow_service("data", "responder", "test_flow", "text-load")
# Verify dispatcher was created with correct parameters
mock_dispatcher_class.assert_called_once_with(
pulsar_client=mock_pulsar_client,
queue={"queue": "text_load_queue"}
)
mock_dispatcher.start.assert_called_once()
mock_dispatcher.process.assert_called_once_with("data", "responder")
# Verify dispatcher was cached
assert manager.dispatchers[("test_flow", "text-load")] == mock_dispatcher
assert result == "sender_result"
@pytest.mark.asyncio
async def test_invoke_flow_service_invalid_flow(self):
"""Test invoke_flow_service with invalid flow"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
with pytest.raises(RuntimeError, match="Invalid flow"):
await manager.invoke_flow_service("data", "responder", "invalid_flow", "agent")
@pytest.mark.asyncio
async def test_invoke_flow_service_unsupported_kind_by_flow(self):
"""Test invoke_flow_service with kind not supported by flow"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
# Setup test flow without agent interface
manager.flows["test_flow"] = {
"interfaces": {
"text-completion": {"request": "req", "response": "resp"}
}
}
with pytest.raises(RuntimeError, match="This kind not supported by flow"):
await manager.invoke_flow_service("data", "responder", "test_flow", "agent")
@pytest.mark.asyncio
async def test_invoke_flow_service_invalid_kind(self):
"""Test invoke_flow_service with invalid kind"""
mock_pulsar_client = Mock()
mock_config_receiver = Mock()
manager = DispatcherManager(mock_pulsar_client, mock_config_receiver)
# Setup test flow with interface but unsupported kind
manager.flows["test_flow"] = {
"interfaces": {
"invalid-kind": {"request": "req", "response": "resp"}
}
}
with patch('trustgraph.gateway.dispatch.manager.request_response_dispatchers') as mock_rr_dispatchers, \
patch('trustgraph.gateway.dispatch.manager.sender_dispatchers') as mock_sender_dispatchers:
mock_rr_dispatchers.__contains__.return_value = False
mock_sender_dispatchers.__contains__.return_value = False
with pytest.raises(RuntimeError, match="Invalid kind"):
await manager.invoke_flow_service("data", "responder", "test_flow", "invalid-kind")