trustgraph/docs/tech-specs/tool-services.md

Tool Services: Dynamically Pluggable Agent Tools

Status

Draft - Gathering Requirements

Overview

This specification defines a mechanism for dynamically pluggable agent tools called "tool services". Unlike the existing built-in tool types (KnowledgeQueryImpl, McpToolImpl, etc.), tool services allow new tools to be introduced by:

1. Deploying a new Pulsar-based service
2. Adding a configuration descriptor that tells the agent how to invoke it

This enables extensibility without modifying the core agent-react framework.

Terminology

Term	Definition
Built-in Tool	Existing tool types with hardcoded implementations in tools.py
Tool Service	A Pulsar service that can be invoked as an agent tool, defined by a service descriptor
Tool	A configured instance that references a tool service, exposed to the agent/LLM

This is a two-tier model, analogous to MCP tools:

• MCP: MCP server defines the tool interface → Tool config references it
• Tool Services: Tool service defines the Pulsar interface → Tool config references it

Current Architecture

Existing Tool Implementation

Tools are currently defined in trustgraph-flow/trustgraph/agent/react/tools.py with typed implementations:

class KnowledgeQueryImpl:
    async def invoke(self, question):
        client = self.context("graph-rag-request")
        return await client.rag(question, self.collection)

Each tool type:

• Has a hardcoded Pulsar service it calls (e.g., graph-rag-request)
• Knows the exact method to call on the client (e.g., client.rag())
• Has typed arguments defined in the implementation

Tool Registration (service.py:105-214)

Tools are loaded from config with a type field that maps to an implementation:

if impl_id == "knowledge-query":
    impl = functools.partial(KnowledgeQueryImpl, collection=data.get("collection"))
elif impl_id == "text-completion":
    impl = TextCompletionImpl
# ... etc

Proposed Architecture

Two-Tier Model

Tier 1: Tool Service Descriptor

A tool service defines a Pulsar service interface. It declares:

• The topic to call
• Configuration parameters it requires from tools that use it

{
  "id": "custom-rag",
  "topic": "custom-rag-request",
  "config-params": [
    {"name": "collection", "required": true}
  ]
}

A tool service that needs no configuration parameters:

{
  "id": "calculator",
  "topic": "calc-request",
  "config-params": []
}

Tier 2: Tool Descriptor

A tool references a tool service and provides:

• Config parameter values (satisfying the service's requirements)
• Tool metadata for the agent (name, description)
• Argument definitions for the LLM

{
  "type": "tool-service",
  "name": "query-customers",
  "description": "Query the customer knowledge base",
  "service": "custom-rag",
  "collection": "customers",
  "arguments": [
    {
      "name": "question",
      "type": "string",
      "description": "The question to ask about customers"
    }
  ]
}

Multiple tools can reference the same service with different configurations:

{
  "type": "tool-service",
  "name": "query-products",
  "description": "Query the product knowledge base",
  "service": "custom-rag",
  "collection": "products",
  "arguments": [
    {
      "name": "question",
      "type": "string",
      "description": "The question to ask about products"
    }
  ]
}

Request Format

When a tool is invoked, the request to the tool service includes:

• user: From the agent request (multi-tenancy)
• Config values: From the tool descriptor (e.g., collection)
• arguments: From the LLM

{
  "user": "alice",
  "collection": "customers",
  "arguments": {
    "question": "What are the top customer complaints?"
  }
}

Generic Tool Service Implementation

A ToolServiceImpl class invokes tool services based on configuration:

class ToolServiceImpl:
    def __init__(self, service_topic, config_values, context):
        self.service_topic = service_topic
        self.config_values = config_values  # e.g., {"collection": "customers"}
        self.context = context

    async def invoke(self, user, **arguments):
        client = self.context(self.service_topic)
        request = {
            "user": user,
            **self.config_values,
            "arguments": arguments,
        }
        response = await client.call(request)
        if isinstance(response, str):
            return response
        else:
            return json.dumps(response)

Design Decisions

Two-Tier Configuration Model

Tool services follow a two-tier model similar to MCP tools:

1. Tool Service: Defines the Pulsar service interface (topic, required config params)
2. Tool: References a tool service, provides config values, defines LLM arguments

This separation allows:

• One tool service to be used by multiple tools with different configurations
• Clear distinction between service interface and tool configuration
• Reusability of service definitions

Request Mapping: Pass-Through with Envelope

The request to a tool service is a structured envelope containing:

• user: Propagated from the agent request for multi-tenancy
• Config values: From the tool descriptor (e.g., collection)
• arguments: LLM-provided arguments, passed through as a dict

The agent manager parses the LLM's response into act.arguments as a dict (agent_manager.py:117-154). This dict is included in the request envelope.

Schema Handling: Untyped

Requests and responses use untyped dicts. No schema validation at the agent level - the tool service is responsible for validating its inputs. This provides maximum flexibility for defining new services.

Client Interface: Direct Pulsar

Tool services are invoked via direct Pulsar messaging, not through the existing typed client abstraction. The tool-service descriptor specifies a Pulsar queue name. A base class will be defined for implementing tool services. Implementation details to be determined during development.

Error Handling: Standard Error Convention

Tool service responses follow the existing schema convention with an error field:

@dataclass
class Error:
    type: str = ""
    message: str = ""

Response structure:

• Success: error is None, response contains result
• Error: error is populated with type and message

This matches the pattern used throughout existing service schemas (e.g., PromptResponse, QueryResponse, AgentResponse).

Request/Response Correlation

Requests and responses are correlated using an id in Pulsar message properties:

• Request includes id in properties: properties={"id": id}
• Response(s) include the same id: properties={"id": id}

This follows the existing pattern used throughout the codebase (e.g., agent_service.py, llm_service.py).

Streaming Support

Tool services can return streaming responses:

• Multiple response messages with the same id in properties
• Each response includes end_of_message: bool field
• Final response has end_of_message: True

This matches the pattern used in AgentResponse and other streaming services.

Response Handling: String Return

All existing tools follow the same pattern: receive arguments as a dict, return observation as a string.

Tool	Response Handling
KnowledgeQueryImpl	Returns client.rag() directly (string)
TextCompletionImpl	Returns client.question() directly (string)
McpToolImpl	Returns string, or json.dumps(output) if not string
StructuredQueryImpl	Formats result to string
PromptImpl	Returns client.prompt() directly (string)

Tool services follow the same contract:

• The service returns a string response (the observation)
• If the response is not a string, it is converted via json.dumps()
• No extraction configuration needed in the descriptor

This keeps the descriptor simple and places responsibility on the service to return an appropriate text response for the agent.

Implementation Considerations

Configuration Structure

Two new config sections:

tool-service/
  custom-rag: {"id": "custom-rag", "topic": "...", "config-params": [...]}
  calculator: {"id": "calculator", "topic": "...", "config-params": []}

tool/
  query-customers: {"type": "tool-service", "service": "custom-rag", ...}
  query-products: {"type": "tool-service", "service": "custom-rag", ...}

Files to Modify

File	Changes
trustgraph-flow/trustgraph/agent/react/tools.py	Add ToolServiceImpl
trustgraph-flow/trustgraph/agent/react/service.py	Load tool-service configs, handle type: "tool-service" in tool configs
trustgraph-base/trustgraph/base/	Add generic client call support

Backward Compatibility

• Existing built-in tool types continue to work unchanged
• tool-service is a new tool type alongside existing types (knowledge-query, mcp-tool, etc.)

Future Considerations

Self-Announcing Services

A future enhancement could allow services to publish their own descriptors:

• Services publish to a well-known tool-descriptors topic on startup
• Agent subscribes and dynamically registers tools
• Enables true plug-and-play without config changes

This is out of scope for the initial implementation.

References

• Current tool implementation: trustgraph-flow/trustgraph/agent/react/tools.py
• Tool registration: trustgraph-flow/trustgraph/agent/react/service.py:105-214
• Agent schemas: trustgraph-base/trustgraph/schema/services/agent.py