trustgraph/docs/tech-specs/tool-services.md

Tool Services: Dynamically Pluggable Agent Tools

Status

Implemented

Overview

This specification defines a mechanism for dynamically pluggable agent tools called "tool services". Unlike the existing built-in tool types (KnowledgeQueryImpl, McpToolImpl, etc.), tool services allow new tools to be introduced by:

1. Deploying a new Pulsar-based service
2. Adding a configuration descriptor that tells the agent how to invoke it

This enables extensibility without modifying the core agent-react framework.

Terminology

Term	Definition
Built-in Tool	Existing tool types with hardcoded implementations in tools.py
Tool Service	A Pulsar service that can be invoked as an agent tool, defined by a service descriptor
Tool	A configured instance that references a tool service, exposed to the agent/LLM

This is a two-tier model, analogous to MCP tools:

• MCP: MCP server defines the tool interface → Tool config references it
• Tool Services: Tool service defines the Pulsar interface → Tool config references it

Background: Existing Tools

Built-in Tool Implementation

Tools are currently defined in trustgraph-flow/trustgraph/agent/react/tools.py with typed implementations:

class KnowledgeQueryImpl:
    async def invoke(self, question):
        client = self.context("graph-rag-request")
        return await client.rag(question, self.collection)

Each tool type:

• Has a hardcoded Pulsar service it calls (e.g., graph-rag-request)
• Knows the exact method to call on the client (e.g., client.rag())
• Has typed arguments defined in the implementation

Tool Registration (service.py:105-214)

Tools are loaded from config with a type field that maps to an implementation:

if impl_id == "knowledge-query":
    impl = functools.partial(KnowledgeQueryImpl, collection=data.get("collection"))
elif impl_id == "text-completion":
    impl = TextCompletionImpl
# ... etc

Architecture

Two-Tier Model

Tier 1: Tool Service Descriptor

A tool service defines a Pulsar service interface. It declares:

• The Pulsar queues for request/response
• Configuration parameters it requires from tools that use it

{
  "id": "custom-rag",
  "request-queue": "non-persistent://tg/request/custom-rag",
  "response-queue": "non-persistent://tg/response/custom-rag",
  "config-params": [
    {"name": "collection", "required": true}
  ]
}

A tool service that needs no configuration parameters:

{
  "id": "calculator",
  "request-queue": "non-persistent://tg/request/calc",
  "response-queue": "non-persistent://tg/response/calc",
  "config-params": []
}

Tier 2: Tool Descriptor

A tool references a tool service and provides:

• Config parameter values (satisfying the service's requirements)
• Tool metadata for the agent (name, description)
• Argument definitions for the LLM

{
  "type": "tool-service",
  "name": "query-customers",
  "description": "Query the customer knowledge base",
  "service": "custom-rag",
  "collection": "customers",
  "arguments": [
    {
      "name": "question",
      "type": "string",
      "description": "The question to ask about customers"
    }
  ]
}

Multiple tools can reference the same service with different configurations:

{
  "type": "tool-service",
  "name": "query-products",
  "description": "Query the product knowledge base",
  "service": "custom-rag",
  "collection": "products",
  "arguments": [
    {
      "name": "question",
      "type": "string",
      "description": "The question to ask about products"
    }
  ]
}

Request Format

When a tool is invoked, the request to the tool service includes:

• user: From the agent request (multi-tenancy)
• config: JSON-encoded config values from the tool descriptor
• arguments: JSON-encoded arguments from the LLM

{
  "user": "alice",
  "config": "{\"collection\": \"customers\"}",
  "arguments": "{\"question\": \"What are the top customer complaints?\"}"
}

The tool service receives these as parsed dicts in the invoke method.

Generic Tool Service Implementation

A ToolServiceImpl class invokes tool services based on configuration:

class ToolServiceImpl:
    def __init__(self, context, request_queue, response_queue, config_values, arguments, processor):
        self.request_queue = request_queue
        self.response_queue = response_queue
        self.config_values = config_values  # e.g., {"collection": "customers"}
        # ...

    async def invoke(self, **arguments):
        client = await self._get_or_create_client()
        response = await client.call(user, self.config_values, arguments)
        if isinstance(response, str):
            return response
        else:
            return json.dumps(response)

Design Decisions

Two-Tier Configuration Model

Tool services follow a two-tier model similar to MCP tools:

1. Tool Service: Defines the Pulsar service interface (topic, required config params)
2. Tool: References a tool service, provides config values, defines LLM arguments

This separation allows:

• One tool service to be used by multiple tools with different configurations
• Clear distinction between service interface and tool configuration
• Reusability of service definitions

Request Mapping: Pass-Through with Envelope

The request to a tool service is a structured envelope containing:

• user: Propagated from the agent request for multi-tenancy
• Config values: From the tool descriptor (e.g., collection)
• arguments: LLM-provided arguments, passed through as a dict

The agent manager parses the LLM's response into act.arguments as a dict (agent_manager.py:117-154). This dict is included in the request envelope.

Schema Handling: Untyped

Requests and responses use untyped dicts. No schema validation at the agent level - the tool service is responsible for validating its inputs. This provides maximum flexibility for defining new services.

Client Interface: Direct Pulsar Topics

Tool services use direct Pulsar topics without requiring flow configuration. The tool-service descriptor specifies the full queue names:

{
  "id": "joke-service",
  "request-queue": "non-persistent://tg/request/joke",
  "response-queue": "non-persistent://tg/response/joke",
  "config-params": [...]
}

This allows services to be hosted in any namespace.

Error Handling: Standard Error Convention

Tool service responses follow the existing schema convention with an error field:

@dataclass
class Error:
    type: str = ""
    message: str = ""

Response structure:

• Success: error is None, response contains result
• Error: error is populated with type and message

This matches the pattern used throughout existing service schemas (e.g., PromptResponse, QueryResponse, AgentResponse).

Request/Response Correlation

Requests and responses are correlated using an id in Pulsar message properties:

• Request includes id in properties: properties={"id": id}
• Response(s) include the same id: properties={"id": id}

This follows the existing pattern used throughout the codebase (e.g., agent_service.py, llm_service.py).

Streaming Support

Tool services can return streaming responses:

• Multiple response messages with the same id in properties
• Each response includes end_of_stream: bool field
• Final response has end_of_stream: True

This matches the pattern used in AgentResponse and other streaming services.

Response Handling: String Return

All existing tools follow the same pattern: receive arguments as a dict, return observation as a string.

Tool	Response Handling
KnowledgeQueryImpl	Returns client.rag() directly (string)
TextCompletionImpl	Returns client.question() directly (string)
McpToolImpl	Returns string, or json.dumps(output) if not string
StructuredQueryImpl	Formats result to string
PromptImpl	Returns client.prompt() directly (string)

Tool services follow the same contract:

• The service returns a string response (the observation)
• If the response is not a string, it is converted via json.dumps()
• No extraction configuration needed in the descriptor

This keeps the descriptor simple and places responsibility on the service to return an appropriate text response for the agent.

Configuration Guide

To add a new tool service, two configuration items are required:

1. Tool Service Configuration

Stored under the tool-service config key. Defines the Pulsar queues and available config parameters.

Field	Required	Description
id	Yes	Unique identifier for the tool service
request-queue	Yes	Full Pulsar topic for requests (e.g., non-persistent://tg/request/joke)
response-queue	Yes	Full Pulsar topic for responses (e.g., non-persistent://tg/response/joke)
config-params	No	Array of config parameters the service accepts

Each config param can specify:

• name: Parameter name (required)
• required: Whether the parameter must be provided by tools (default: false)

Example:

{
  "id": "joke-service",
  "request-queue": "non-persistent://tg/request/joke",
  "response-queue": "non-persistent://tg/response/joke",
  "config-params": [
    {"name": "style", "required": false}
  ]
}

2. Tool Configuration

Stored under the tool config key. Defines a tool that the agent can use.

Field	Required	Description
type	Yes	Must be "tool-service"
name	Yes	Tool name exposed to the LLM
description	Yes	Description of what the tool does (shown to LLM)
service	Yes	ID of the tool-service to invoke
arguments	No	Array of argument definitions for the LLM
(config params)	Varies	Any config params defined by the service

Each argument can specify:

• name: Argument name (required)
• type: Data type, e.g., "string" (required)
• description: Description shown to the LLM (required)

Example:

{
  "type": "tool-service",
  "name": "tell-joke",
  "description": "Tell a joke on a given topic",
  "service": "joke-service",
  "style": "pun",
  "arguments": [
    {
      "name": "topic",
      "type": "string",
      "description": "The topic for the joke (e.g., programming, animals, food)"
    }
  ]
}

Loading Configuration

Use tg-put-config-item to load configurations:

# Load tool-service config
tg-put-config-item tool-service/joke-service < joke-service.json

# Load tool config
tg-put-config-item tool/tell-joke < tell-joke.json

The agent-manager must be restarted to pick up new configurations.

Implementation Details

Schema

Request and response types in trustgraph-base/trustgraph/schema/services/tool_service.py:

@dataclass
class ToolServiceRequest:
    user: str = ""           # User context for multi-tenancy
    config: str = ""         # JSON-encoded config values from tool descriptor
    arguments: str = ""      # JSON-encoded arguments from LLM

@dataclass
class ToolServiceResponse:
    error: Error | None = None
    response: str = ""       # String response (the observation)
    end_of_stream: bool = False

Server-Side: DynamicToolService

Base class in trustgraph-base/trustgraph/base/dynamic_tool_service.py:

class DynamicToolService(AsyncProcessor):
    """Base class for implementing tool services."""

    def __init__(self, **params):
        topic = params.get("topic", default_topic)
        # Constructs topics: non-persistent://tg/request/{topic}, non-persistent://tg/response/{topic}
        # Sets up Consumer and Producer

    async def invoke(self, user, config, arguments):
        """Override this method to implement the tool's logic."""
        raise NotImplementedError()

Client-Side: ToolServiceImpl

Implementation in trustgraph-flow/trustgraph/agent/react/tools.py:

class ToolServiceImpl:
    def __init__(self, context, request_queue, response_queue, config_values, arguments, processor):
        # Uses the provided queue paths directly
        # Creates ToolServiceClient on first use

    async def invoke(self, **arguments):
        client = await self._get_or_create_client()
        response = await client.call(user, config_values, arguments)
        return response if isinstance(response, str) else json.dumps(response)

Files

File	Purpose
trustgraph-base/trustgraph/schema/services/tool_service.py	Request/response schemas
trustgraph-base/trustgraph/base/tool_service_client.py	Client for invoking services
trustgraph-base/trustgraph/base/dynamic_tool_service.py	Base class for service implementation
trustgraph-flow/trustgraph/agent/react/tools.py	ToolServiceImpl class
trustgraph-flow/trustgraph/agent/react/service.py	Config loading

Example: Joke Service

An example service in trustgraph-flow/trustgraph/tool_service/joke/:

class Processor(DynamicToolService):
    async def invoke(self, user, config, arguments):
        style = config.get("style", "pun")
        topic = arguments.get("topic", "")
        joke = pick_joke(topic, style)
        return f"Hey {user}! Here's a {style} for you:\n\n{joke}"

Tool service config:

{
  "id": "joke-service",
  "request-queue": "non-persistent://tg/request/joke",
  "response-queue": "non-persistent://tg/response/joke",
  "config-params": [{"name": "style", "required": false}]
}

Tool config:

{
  "type": "tool-service",
  "name": "tell-joke",
  "description": "Tell a joke on a given topic",
  "service": "joke-service",
  "style": "pun",
  "arguments": [
    {"name": "topic", "type": "string", "description": "The topic for the joke"}
  ]
}

Backward Compatibility

• Existing built-in tool types continue to work unchanged
• tool-service is a new tool type alongside existing types (knowledge-query, mcp-tool, etc.)

Future Considerations

Self-Announcing Services

A future enhancement could allow services to publish their own descriptors:

• Services publish to a well-known tool-descriptors topic on startup
• Agent subscribes and dynamically registers tools
• Enables true plug-and-play without config changes

This is out of scope for the initial implementation.

References

• Current tool implementation: trustgraph-flow/trustgraph/agent/react/tools.py
• Tool registration: trustgraph-flow/trustgraph/agent/react/service.py:105-214
• Agent schemas: trustgraph-base/trustgraph/schema/services/agent.py