mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-04-25 00:16:23 +02:00
Tech spec first pass
This commit is contained in:
Cyber MacGeddon
parent
e5b9b4976a
commit
c28569fc73
1 changed files with 354 additions and 0 deletions
354
docs/tech-specs/multi-agent-planning-framework.md
Normal file
354
docs/tech-specs/multi-agent-planning-framework.md
Normal file
|
|
@ -0,0 +1,354 @@
|
|||
# TechSpec: Multi-Agent Planning Framework (MAPF)
|
||||
|
||||
## Overview
|
||||
|
||||
The Multi-Agent Planning Framework (MAPF) is a confidence-driven agent architecture that complements the existing ReAct framework. While ReAct uses iterative reasoning cycles, MAPF generates upfront execution plans with confidence thresholds and executes them through specialized microservices.
|
||||
|
||||
## Architecture Design
|
||||
|
||||
### Core Philosophy
|
||||
- **Confidence-Driven Execution**: Each step has confidence thresholds that determine progression
|
||||
- **Upfront Planning**: Generate complete execution plans before execution begins
|
||||
- **Specialized Functions**: Domain-specific microservices for different operation types
|
||||
- **Fault Tolerance**: Retry mechanisms and fallback strategies based on confidence scores
|
||||
|
||||
### System Components
|
||||
|
||||
#### 1. Planning Service (`trustgraph-mapf-planning`)
|
||||
|
||||
**Responsibility**: Generate execution plans from user requests
|
||||
|
||||
**Input Schema**:
|
||||
```python
|
||||
@dataclasses.dataclass
|
||||
class PlanRequest:
|
||||
user_request: str
|
||||
context: Optional[dict] = None
|
||||
constraints: Optional[dict] = None
|
||||
```
|
||||
|
||||
**Output Schema**:
|
||||
```python
|
||||
@dataclasses.dataclass
|
||||
class ExecutionPlan:
|
||||
plan_id: str
|
||||
steps: List[PlanStep]
|
||||
confidence_threshold: float = 0.7
|
||||
max_retries: int = 3
|
||||
|
||||
@dataclasses.dataclass
|
||||
class PlanStep:
|
||||
step_id: str
|
||||
function_type: str # query, external_query, compare, evaluate, etc.
|
||||
parameters: dict
|
||||
confidence_threshold: float
|
||||
dependencies: List[str] # Previous step IDs this depends on
|
||||
retry_strategy: Optional[dict] = None
|
||||
```
|
||||
|
||||
**Implementation**:
|
||||
- Uses LLM to decompose complex requests into structured plans
|
||||
- Assigns confidence thresholds based on task complexity
|
||||
- Determines step dependencies and execution order
|
||||
|
||||
#### 2. Flow Controller Service (`trustgraph-mapf-controller`)
|
||||
|
||||
**Responsibility**: Orchestrate plan execution across microservices
|
||||
|
||||
**Key Features**:
|
||||
- Step dependency resolution
|
||||
- Confidence score validation
|
||||
- Retry logic and failure handling
|
||||
- Context management between steps
|
||||
|
||||
**Execution Logic**:
|
||||
```python
|
||||
async def execute_plan(self, plan: ExecutionPlan):
|
||||
context = ExecutionContext(plan_id=plan.plan_id)
|
||||
|
||||
for step in plan.steps:
|
||||
if not self.dependencies_satisfied(step, context):
|
||||
await self.wait_for_dependencies(step, context)
|
||||
|
||||
result = await self.execute_step(step, context)
|
||||
|
||||
if result.confidence < step.confidence_threshold:
|
||||
if context.retries[step.step_id] < plan.max_retries:
|
||||
await self.retry_step(step, context)
|
||||
else:
|
||||
await self.handle_failure(step, context)
|
||||
else:
|
||||
context.add_result(step.step_id, result)
|
||||
```
|
||||
|
||||
#### 3. Function Services
|
||||
|
||||
Specialized microservices for different operation types:
|
||||
|
||||
##### Query Service (`trustgraph-mapf-query`)
|
||||
- Internal knowledge base queries
|
||||
- Uses existing graph RAG infrastructure
|
||||
- Handles collection-specific queries
|
||||
|
||||
##### External Query Service (`trustgraph-mapf-external`)
|
||||
- Web searches, API calls
|
||||
- Rate limiting and error handling
|
||||
- Response validation and confidence scoring
|
||||
|
||||
##### Processing Services:
|
||||
- **Compare Service** (`trustgraph-mapf-compare`): Data comparison operations
|
||||
- **Evaluate Service** (`trustgraph-mapf-evaluate`): Assessment and scoring
|
||||
- **Compute Service** (`trustgraph-mapf-compute`): Mathematical operations
|
||||
- **Filter Service** (`trustgraph-mapf-filter`): Data filtering and selection
|
||||
- **Prioritize Service** (`trustgraph-mapf-prioritize`): Ranking and ordering
|
||||
- **Deduce Service** (`trustgraph-mapf-deduce`): Logical inference
|
||||
|
||||
##### Document Service (`trustgraph-mapf-document`)
|
||||
- Final output generation
|
||||
- Report compilation
|
||||
- Format conversion
|
||||
|
||||
#### 4. Data Store Service (`trustgraph-mapf-datastore`)
|
||||
|
||||
**Responsibility**: Execution context and intermediate results
|
||||
|
||||
**Features**:
|
||||
- Step result storage
|
||||
- Context sharing between services
|
||||
- Execution history and audit trails
|
||||
|
||||
### Message Flow Architecture
|
||||
|
||||
#### Pulsar Topics:
|
||||
- `mapf-plan-requests`
|
||||
- `mapf-execution-plans`
|
||||
- `mapf-step-requests`
|
||||
- `mapf-step-responses`
|
||||
- `mapf-completion-notices`
|
||||
|
||||
#### Message Schemas:
|
||||
|
||||
```python
|
||||
@dataclasses.dataclass
|
||||
class StepRequest:
|
||||
step_id: str
|
||||
plan_id: str
|
||||
function_type: str
|
||||
parameters: dict
|
||||
context: dict
|
||||
timeout: int = 300
|
||||
|
||||
@dataclasses.dataclass
|
||||
class StepResponse:
|
||||
step_id: str
|
||||
plan_id: str
|
||||
success: bool
|
||||
result: Any
|
||||
confidence: float
|
||||
execution_time: float
|
||||
error: Optional[str] = None
|
||||
```
|
||||
|
||||
### Integration with Existing Infrastructure
|
||||
|
||||
#### Shared Components:
|
||||
- **Pulsar Messaging**: Reuse existing pub/sub infrastructure
|
||||
- **Graph RAG**: Query service leverages existing knowledge graph
|
||||
- **Prompt Templates**: Planning service uses existing template system
|
||||
- **Configuration**: Extend existing config management
|
||||
|
||||
#### Service Registration:
|
||||
```python
|
||||
# In service.py for each MAPF service
|
||||
class MapfPlanningService(BaseService):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.register_specification(
|
||||
PlanRequestSpec(
|
||||
request_name="mapf-plan-request",
|
||||
response_name="mapf-execution-plan"
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
### Configuration Schema
|
||||
|
||||
```yaml
|
||||
mapf:
|
||||
planning:
|
||||
model: "gpt-4"
|
||||
default_confidence_threshold: 0.7
|
||||
max_plan_complexity: 20
|
||||
|
||||
execution:
|
||||
max_concurrent_steps: 5
|
||||
step_timeout: 300
|
||||
retry_backoff: exponential
|
||||
|
||||
functions:
|
||||
query:
|
||||
collections: ["default", "research", "technical"]
|
||||
external:
|
||||
rate_limit: 10 # requests per minute
|
||||
timeout: 30
|
||||
```
|
||||
|
||||
### Deployment Considerations
|
||||
|
||||
#### Docker Services:
|
||||
- Each function service as separate container
|
||||
- Shared base image with common utilities
|
||||
- Horizontal scaling based on load
|
||||
|
||||
#### Development Workflow:
|
||||
1. Add new function services in `trustgraph-mapf-functions/`
|
||||
2. Implement service interface and Pulsar specs
|
||||
3. Add to docker-compose and deployment scripts
|
||||
4. Create integration tests
|
||||
|
||||
### Comparison with ReAct
|
||||
|
||||
| Aspect | ReAct | MAPF |
|
||||
|--------|-------|------|
|
||||
| Planning | Iterative, step-by-step | Upfront, complete plan |
|
||||
| Execution | Sequential reasoning cycles | Parallel, dependency-based |
|
||||
| Error Handling | Human-readable error messages | Confidence-based retries |
|
||||
| Use Cases | Interactive problem solving | Batch processing, complex workflows |
|
||||
| LLM Usage | High (every iteration) | Medium (planning phase) |
|
||||
|
||||
### Implementation Phases
|
||||
|
||||
#### Phase 1: Core Infrastructure
|
||||
- Planning service with basic plan generation
|
||||
- Flow controller with sequential execution
|
||||
- Query service integration
|
||||
- Basic Pulsar messaging
|
||||
|
||||
#### Phase 2: Function Services
|
||||
- Implement processing services (compare, evaluate, etc.)
|
||||
- External query service with web search
|
||||
- Document service for output generation
|
||||
- Confidence scoring mechanisms
|
||||
|
||||
#### Phase 3: Advanced Features
|
||||
- Parallel execution optimization
|
||||
- Dynamic plan modification
|
||||
- Performance monitoring and analytics
|
||||
- Integration with existing agent tools
|
||||
|
||||
### Success Metrics
|
||||
|
||||
- **Plan Success Rate**: % of plans that complete successfully
|
||||
- **Confidence Accuracy**: How well confidence scores predict success
|
||||
- **Execution Time**: Average time from request to completion
|
||||
- **Resource Utilization**: Service load balancing and efficiency
|
||||
- **Error Recovery**: Success rate of retry mechanisms
|
||||
|
||||
### Example Plan Generation
|
||||
|
||||
For a request like "Compare the top 3 machine learning frameworks and recommend the best for a startup":
|
||||
|
||||
```json
|
||||
{
|
||||
"plan_id": "plan-12345",
|
||||
"confidence_threshold": 0.7,
|
||||
"max_retries": 3,
|
||||
"steps": [
|
||||
{
|
||||
"step_id": "step-1",
|
||||
"function_type": "query",
|
||||
"parameters": {
|
||||
"question": "What are the top machine learning frameworks?",
|
||||
"collection": "technical"
|
||||
},
|
||||
"confidence_threshold": 0.8,
|
||||
"dependencies": []
|
||||
},
|
||||
{
|
||||
"step_id": "step-2",
|
||||
"function_type": "external_query",
|
||||
"parameters": {
|
||||
"query": "latest ML framework popularity statistics 2024",
|
||||
"sources": ["web", "github"]
|
||||
},
|
||||
"confidence_threshold": 0.6,
|
||||
"dependencies": []
|
||||
},
|
||||
{
|
||||
"step_id": "step-3",
|
||||
"function_type": "filter",
|
||||
"parameters": {
|
||||
"input": "${step-1.result}",
|
||||
"criteria": "top_3_by_popularity",
|
||||
"additional_data": "${step-2.result}"
|
||||
},
|
||||
"confidence_threshold": 0.7,
|
||||
"dependencies": ["step-1", "step-2"]
|
||||
},
|
||||
{
|
||||
"step_id": "step-4",
|
||||
"function_type": "compare",
|
||||
"parameters": {
|
||||
"items": "${step-3.result}",
|
||||
"criteria": ["ease_of_use", "community_support", "performance", "cost"]
|
||||
},
|
||||
"confidence_threshold": 0.7,
|
||||
"dependencies": ["step-3"]
|
||||
},
|
||||
{
|
||||
"step_id": "step-5",
|
||||
"function_type": "evaluate",
|
||||
"parameters": {
|
||||
"comparison": "${step-4.result}",
|
||||
"context": "startup with limited resources"
|
||||
},
|
||||
"confidence_threshold": 0.75,
|
||||
"dependencies": ["step-4"]
|
||||
},
|
||||
{
|
||||
"step_id": "step-6",
|
||||
"function_type": "document",
|
||||
"parameters": {
|
||||
"type": "recommendation_report",
|
||||
"evaluation": "${step-5.result}",
|
||||
"comparison": "${step-4.result}",
|
||||
"format": "markdown"
|
||||
},
|
||||
"confidence_threshold": 0.8,
|
||||
"dependencies": ["step-5"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### API Endpoints
|
||||
|
||||
The framework will expose the following REST API endpoints through the gateway:
|
||||
|
||||
- `POST /mapf/plan` - Submit a new planning request
|
||||
- `GET /mapf/plan/{plan_id}` - Get plan status and details
|
||||
- `GET /mapf/plan/{plan_id}/steps/{step_id}` - Get specific step results
|
||||
- `POST /mapf/plan/{plan_id}/retry` - Retry a failed plan
|
||||
- `DELETE /mapf/plan/{plan_id}` - Cancel an executing plan
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
#### Unit Tests:
|
||||
- Planning service plan generation
|
||||
- Individual function service logic
|
||||
- Confidence scoring algorithms
|
||||
- Message serialization/deserialization
|
||||
|
||||
#### Integration Tests:
|
||||
- End-to-end plan execution
|
||||
- Service communication via Pulsar
|
||||
- Error handling and retry logic
|
||||
- Context persistence
|
||||
|
||||
#### Performance Tests:
|
||||
- Plan execution throughput
|
||||
- Service scaling behavior
|
||||
- Message queue performance
|
||||
- Resource utilization
|
||||
|
||||
This framework provides a complementary approach to the ReAct system, offering structured planning and execution for complex, multi-step tasks while leveraging the existing TrustGraph infrastructure.
|
||||
Loading…
Add table
Add a link
Reference in a new issue