mirror of
https://github.com/katanemo/plano.git
synced 2026-06-17 15:25:17 +02:00
650 lines
26 KiB
ReStructuredText
650 lines
26 KiB
ReStructuredText
.. _agent_routing:
|
|
|
|
Orchestration
|
|
==============
|
|
|
|
Building multi-agent systems allow you to route requests across multiple specialized agents, each designed to handle specific types of tasks.
|
|
Plano makes it easy to build and scale these systems by managing the orchestration layer—deciding which agent(s) should handle each request—while you focus on implementing individual agent logic.
|
|
|
|
This guide shows you how to configure and implement multi-agent orchestration in Plano using a real-world example: a **Travel Booking Assistant** that routes queries to specialized agents for weather, flights, and currency exchange.
|
|
|
|
How It Works
|
|
------------
|
|
|
|
Plano's orchestration layer analyzes incoming prompts and routes them to the most appropriate agent based on user intent and conversation context. The workflow is:
|
|
|
|
1. **User submits a prompt**: The request arrives at Plano's agent listener.
|
|
2. **Agent selection**: Plano analyzes the prompt to determine user intent and complexity, and routes the request to the most suitable agent configured in your system—such as a weather agent, flight agent, or currency agent.
|
|
3. **Agent handles request**: The selected agent processes the prompt using its specialized logic and tools.
|
|
4. **Seamless handoffs**: For multi-turn conversations, Plano repeats the intent analysis for each follow-up query, enabling smooth handoffs between agents as the conversation evolves.
|
|
|
|
Example: Travel Booking Assistant
|
|
----------------------------------
|
|
|
|
Let's walk through a complete multi-agent system: a Travel Booking Assistant that helps users plan trips by providing weather forecasts, flight information, and currency exchange rates. This system uses three specialized agents:
|
|
|
|
* **Weather Agent**: Provides real-time weather conditions and forecasts
|
|
* **Flight Agent**: Searches for flights between airports
|
|
* **Currency Agent**: Provides currency exchange rates and conversions
|
|
|
|
Configuration
|
|
-------------
|
|
|
|
Configure your agents in the ``listeners`` section of your ``plano_config.yaml``:
|
|
|
|
.. code-block:: yaml
|
|
:caption: Travel Booking Multi-Agent Configuration
|
|
|
|
version: v0.3.0
|
|
|
|
agents:
|
|
- id: weather_agent
|
|
url: http://host.docker.internal:10510
|
|
- id: flight_agent
|
|
url: http://host.docker.internal:10520
|
|
- id: currency_agent
|
|
url: http://host.docker.internal:10530
|
|
|
|
model_providers:
|
|
- model: openai/gpt-4o
|
|
access_key: $OPENAI_API_KEY
|
|
- model: openai/gpt-4o-mini
|
|
access_key: $OPENAI_API_KEY
|
|
|
|
listeners:
|
|
- type: agent
|
|
name: travel_booking_service
|
|
port: 8001
|
|
router: plano_orchestrator_v1
|
|
agents:
|
|
- id: weather_agent
|
|
description: Get real-time weather conditions and multi-day forecasts for any city worldwide using Open-Meteo API (free, no API key needed). Provides current temperature, multi-day forecasts, weather conditions, sunrise/sunset times, and detailed weather information. Understands conversation context to resolve location references from previous messages.
|
|
|
|
- id: flight_agent
|
|
description: Get live flight information between airports using FlightAware AeroAPI. Shows real-time flight status, scheduled/estimated/actual departure and arrival times, gate and terminal information, delays, aircraft type, and flight status. Automatically resolves city names to airport codes (IATA/ICAO). Understands conversation context to infer origin/destination from follow-up questions. Supports queries like "What flights go from London to Seattle?" or "Do they fly out from Seattle?" (using context from previous messages).
|
|
|
|
- id: currency_agent
|
|
description: Get real-time currency exchange rates and perform currency conversions using Frankfurter API (free, no API key needed). Provides latest exchange rates, currency conversions with amount calculations, and supports any currency pair. Automatically extracts currency codes from country names and conversation context. Understands pronouns like "their currency" when referring to previously mentioned countries. Uses standard 3-letter ISO currency codes (e.g., USD, EUR, GBP, JPY, PKR).
|
|
|
|
tracing:
|
|
random_sampling: 100
|
|
|
|
**Key Configuration Elements:**
|
|
|
|
* **agent listener**: A listener of ``type: agent`` tells Plano to perform intent analysis and routing for incoming requests.
|
|
* **agents list**: Define each agent with an ``id``, ``description`` (used for routing decisions)
|
|
* **router**: The ``plano_orchestrator_v1`` router uses Plano-Orchestrator to analyze user intent and select the appropriate agent.
|
|
* **filter_chain**: Optionally attach :ref:`filter chains <filter_chain>` to agents for guardrails, query rewriting, or context enrichment.
|
|
|
|
**Writing Effective Agent Descriptions**
|
|
|
|
Agent descriptions are critical—they're used by Plano-Orchestrator to make routing decisions. Be specific about:
|
|
|
|
* What the agent does (e.g., "Get real-time weather conditions")
|
|
* What APIs or tools it uses (e.g., "using Open-Meteo API")
|
|
* What information it provides (e.g., "current temperature, multi-day forecasts")
|
|
* How it handles context (e.g., "Understands conversation context to resolve location references")
|
|
|
|
Implementation
|
|
--------------
|
|
|
|
Agents are HTTP services that receive routed requests from Plano. Each agent implements the OpenAI Chat Completions API format, making them compatible with standard LLM clients.
|
|
|
|
Agent Structure
|
|
^^^^^^^^^^^^^^^
|
|
|
|
Let's examine the Weather Agent implementation:
|
|
|
|
.. code-block:: python
|
|
:caption: Weather Agent - Core Structure
|
|
|
|
from fastapi import FastAPI, Request
|
|
from fastapi.responses import StreamingResponse
|
|
from openai import AsyncOpenAI
|
|
import os
|
|
import httpx
|
|
from .api import ChatCompletionRequest, ChatCompletionStreamResponse
|
|
|
|
# Configuration
|
|
PLANO_ENDPOINT = os.getenv("PLANO_ENDPOINT", "http://localhost:12000/v1")
|
|
WEATHER_MODEL = "openai/gpt-4o"
|
|
LOCATION_MODEL = "openai/gpt-4o-mini"
|
|
|
|
# Initialize OpenAI client for Plano
|
|
plano = AsyncOpenAI(
|
|
base_url=PLANO_ENDPOINT,
|
|
api_key="EMPTY",
|
|
)
|
|
|
|
app = FastAPI(title="Weather Forecast Agent")
|
|
|
|
@app.post("/v1/chat/completions")
|
|
async def chat_completion_http(request: Request, request_body: ChatCompletionRequest):
|
|
"""HTTP endpoint for chat completions with streaming support."""
|
|
return StreamingResponse(
|
|
stream_chat_completions(request_body),
|
|
media_type="text/event-stream",
|
|
)
|
|
|
|
**Key Points:**
|
|
|
|
* Agents expose a ``/v1/chat/completions`` endpoint that matches OpenAI's API format
|
|
* They use Plano's LLM gateway (via ``PLANO_ENDPOINT``) for all LLM calls
|
|
* They receive the full conversation history in ``request_body.messages``
|
|
|
|
Information Extraction with LLMs
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Agents use LLMs to extract structured information from natural language queries. This enables them to understand user intent and extract parameters needed for API calls.
|
|
|
|
The Weather Agent extracts location information:
|
|
|
|
.. code-block:: python
|
|
:caption: Weather Agent - Location Extraction
|
|
|
|
LOCATION_EXTRACTION_PROMPT = """You are a location extraction assistant. Your ONLY job is to extract the geographic location (city, state, country, etc.) from user messages.
|
|
|
|
CRITICAL RULES:
|
|
1. Extract ONLY the location name - nothing else
|
|
2. Return just the location name in plain text (e.g., "London", "New York", "Paris, France")
|
|
3. If the user mentions multiple locations, extract the PRIMARY location they're asking about
|
|
4. Ignore error messages, HTML tags, and assistant responses
|
|
5. If no clear location is found, return exactly: "NOT_FOUND"
|
|
"""
|
|
|
|
async def extract_location_from_messages(messages):
|
|
"""Extract location from user messages using LLM."""
|
|
user_messages = [msg for msg in messages if msg.role == "user"]
|
|
|
|
if not user_messages:
|
|
return "New York" # Default fallback
|
|
|
|
# Get the most recent user message
|
|
user_content = user_messages[-1].content.strip()
|
|
|
|
response = await plano.chat.completions.create(
|
|
model=LOCATION_MODEL,
|
|
messages=[
|
|
{"role": "system", "content": LOCATION_EXTRACTION_PROMPT},
|
|
{"role": "user", "content": user_content},
|
|
],
|
|
temperature=0.1,
|
|
max_tokens=50,
|
|
)
|
|
|
|
location = response.choices[0].message.content.strip()
|
|
return location if location != "NOT_FOUND" else "New York"
|
|
|
|
The Flight Agent extracts more complex information—origin, destination, and dates:
|
|
|
|
.. code-block:: python
|
|
:caption: Flight Agent - Flight Information Extraction
|
|
|
|
FLIGHT_EXTRACTION_PROMPT = """You are a flight information extraction assistant. Extract flight-related information from user messages and convert it to structured data.
|
|
|
|
CRITICAL RULES:
|
|
1. Extract origin city/airport and destination city/airport from the message AND conversation context
|
|
2. Extract any mentioned dates or time references
|
|
3. PAY ATTENTION TO CONVERSATION CONTEXT - THIS IS CRITICAL:
|
|
- If previous messages mention cities/countries, use that context to resolve pronouns and incomplete queries
|
|
- Example: Previous: "What's the weather in Istanbul?" → Current: "Do they fly out from Seattle?"
|
|
→ This likely means: origin=Istanbul, destination=Seattle
|
|
4. Return your response as a JSON object:
|
|
{
|
|
"origin": "London" or null,
|
|
"destination": "Seattle" or null,
|
|
"date": "2025-12-20" or null,
|
|
"origin_airport_code": "LHR" or null,
|
|
"destination_airport_code": "SEA" or null
|
|
}
|
|
"""
|
|
|
|
async def extract_flight_info_from_messages(messages):
|
|
"""Extract flight information from user messages using LLM, considering conversation context."""
|
|
# Build conversation context from all messages
|
|
conversation_context = []
|
|
for msg in messages:
|
|
content = msg.content.strip()
|
|
# Skip error messages and HTML tags
|
|
if not any(pattern in content.lower() for pattern in ["<", ">", "error:"]):
|
|
conversation_context.append({"role": msg.role, "content": content})
|
|
|
|
# Use last 10 messages for context
|
|
context_messages = conversation_context[-10:] if len(conversation_context) > 10 else conversation_context
|
|
|
|
llm_messages = [
|
|
{"role": "system", "content": FLIGHT_EXTRACTION_PROMPT}
|
|
] + context_messages
|
|
|
|
response = await plano.chat.completions.create(
|
|
model=FLIGHT_EXTRACTION_MODEL,
|
|
messages=llm_messages,
|
|
temperature=0.1,
|
|
max_tokens=300,
|
|
)
|
|
|
|
extracted_text = response.choices[0].message.content.strip()
|
|
# Parse JSON from response
|
|
flight_info = json.loads(extracted_text)
|
|
return flight_info
|
|
|
|
**Key Points:**
|
|
|
|
* Use smaller, faster models (like ``gpt-4o-mini``) for extraction tasks
|
|
* Include conversation context to handle follow-up questions and pronouns
|
|
* Use structured prompts with clear output formats (JSON)
|
|
* Handle edge cases with fallback values
|
|
|
|
Calling External APIs
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
After extracting information, agents call external APIs to fetch real-time data:
|
|
|
|
.. code-block:: python
|
|
:caption: Weather Agent - External API Call
|
|
|
|
async def get_weather_data(location: str, days: int = 1):
|
|
"""Get live weather data for a location using Open-Meteo API."""
|
|
# Step 1: Geocode location to get coordinates
|
|
geocode_result = await geocode_city(location)
|
|
if not geocode_result:
|
|
return {"error": "Could not find location"}
|
|
|
|
latitude = geocode_result["latitude"]
|
|
longitude = geocode_result["longitude"]
|
|
|
|
# Step 2: Fetch weather data
|
|
url = (
|
|
f"https://api.open-meteo.com/v1/forecast?"
|
|
f"latitude={latitude}&"
|
|
f"longitude={longitude}&"
|
|
f"daily=temperature_2m_max,temperature_2m_min,weather_code&"
|
|
f"forecast_days={days}&"
|
|
f"timezone=auto"
|
|
)
|
|
|
|
response = await http_client.get(url)
|
|
weather_data = response.json()
|
|
|
|
# Step 3: Transform API response into structured format
|
|
forecast = []
|
|
for i in range(days):
|
|
forecast.append({
|
|
"date": daily_data["time"][i],
|
|
"temperature_max_c": daily_data["temperature_2m_max"][i],
|
|
"temperature_min_c": daily_data["temperature_2m_min"][i],
|
|
"condition": weather_code_to_condition(daily_data["weather_code"][i]),
|
|
})
|
|
|
|
return {"location": location, "forecast": forecast}
|
|
|
|
The Flight Agent calls FlightAware's AeroAPI:
|
|
|
|
.. code-block:: python
|
|
:caption: Flight Agent - External API Call
|
|
|
|
async def get_flights_between_airports(
|
|
origin_code: str, dest_code: str, start_date: str = None
|
|
):
|
|
"""Get flights between two airports using FlightAware AeroAPI."""
|
|
url = f"{AEROAPI_BASE_URL}/airports/{origin_code}/flights/to/{dest_code}"
|
|
headers = {"x-apikey": AEROAPI_KEY}
|
|
|
|
params = {"connection": "nonstop", "max_pages": 1}
|
|
if start_date:
|
|
params["start"] = start_date
|
|
|
|
response = await http_client.get(url, headers=headers, params=params)
|
|
data = response.json()
|
|
|
|
# Transform API response
|
|
flights = []
|
|
for flight_group in data.get("flights", []):
|
|
segment = flight_group.get("segments", [])[0]
|
|
flights.append({
|
|
"ident": segment.get("ident"),
|
|
"operator": segment.get("operator"),
|
|
"origin": segment.get("origin", {}).get("city"),
|
|
"destination": segment.get("destination", {}).get("city"),
|
|
"scheduled_out": segment.get("scheduled_out"),
|
|
"status": segment.get("status"),
|
|
})
|
|
|
|
return {"flights": flights}
|
|
|
|
**Key Points:**
|
|
|
|
* Use async HTTP clients (like ``httpx.AsyncClient``) for non-blocking API calls
|
|
* Transform external API responses into consistent, structured formats
|
|
* Handle errors gracefully with fallback values
|
|
* Cache or validate data when appropriate (e.g., airport code validation)
|
|
|
|
Preparing Context and Generating Responses
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Agents combine extracted information, API data, and conversation history to generate responses:
|
|
|
|
.. code-block:: python
|
|
:caption: Weather Agent - Context Preparation and Response Generation
|
|
|
|
SYSTEM_PROMPT = """You are a professional weather information assistant. Your role is to provide accurate, clear, and helpful weather information based on the structured weather data provided to you.
|
|
|
|
CRITICAL INSTRUCTIONS:
|
|
1. You will receive weather data as JSON in a system message
|
|
2. Present temperatures in both Celsius and Fahrenheit when available
|
|
3. Use natural, conversational language
|
|
4. Never invent or guess weather data - only use what's provided
|
|
"""
|
|
|
|
async def prepare_weather_messages(request_body: ChatCompletionRequest):
|
|
"""Prepare messages with weather data."""
|
|
# Step 1: Extract location from conversation
|
|
location = await extract_location_from_messages(request_body.messages)
|
|
|
|
# Step 2: Determine if user wants forecast (multi-day)
|
|
last_user_msg = ""
|
|
for msg in reversed(request_body.messages):
|
|
if msg.role == "user":
|
|
last_user_msg = msg.content.lower()
|
|
break
|
|
|
|
days = 5 if "forecast" in last_user_msg or "week" in last_user_msg else 1
|
|
|
|
# Step 3: Fetch weather data
|
|
weather_data = await get_weather_data(location, days)
|
|
|
|
# Step 4: Build context message with structured data
|
|
weather_context = f"""
|
|
Current weather data for {weather_data['location']}:
|
|
|
|
{json.dumps(weather_data, indent=2)}
|
|
|
|
Use this data to answer the user's weather query.
|
|
"""
|
|
|
|
# Step 5: Combine system prompt, context, and conversation history
|
|
response_messages = [
|
|
{"role": "system", "content": SYSTEM_PROMPT},
|
|
{"role": "system", "content": weather_context},
|
|
]
|
|
|
|
# Add conversation history
|
|
for msg in request_body.messages:
|
|
response_messages.append({"role": msg.role, "content": msg.content})
|
|
|
|
return response_messages
|
|
|
|
async def stream_chat_completions(request_body: ChatCompletionRequest):
|
|
"""Generate streaming chat completions."""
|
|
# Prepare messages with weather data
|
|
response_messages = await prepare_weather_messages(request_body)
|
|
|
|
# Call Plano's LLM gateway
|
|
response_stream = await plano.chat.completions.create(
|
|
model=WEATHER_MODEL,
|
|
messages=response_messages,
|
|
temperature=request_body.temperature or 0.7,
|
|
max_tokens=request_body.max_tokens or 1000,
|
|
stream=True,
|
|
)
|
|
|
|
# Stream response chunks
|
|
async for chunk in response_stream:
|
|
if chunk.choices and chunk.choices[0].delta.content:
|
|
content = chunk.choices[0].delta.content
|
|
yield f"data: {chunk.model_dump_json()}\n\n"
|
|
|
|
yield "data: [DONE]\n\n"
|
|
|
|
**Key Points:**
|
|
|
|
* Use system messages to provide structured data to the LLM
|
|
* Include full conversation history for context-aware responses
|
|
* Stream responses for better user experience
|
|
* Route all LLM calls through Plano's gateway for consistent behavior and observability
|
|
|
|
Handling Conversation Context
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
One of the most powerful features of multi-agent systems is handling follow-up questions that reference previous context. The Flight Agent demonstrates this:
|
|
|
|
.. code-block:: python
|
|
:caption: Flight Agent - Context-Aware Follow-up Questions
|
|
|
|
# User: "What's the weather in Istanbul?"
|
|
# Plano routes to weather_agent → Response: "Istanbul is sunny, 25°C"
|
|
|
|
# User: "Do they fly out from Seattle?"
|
|
# Plano routes to flight_agent → Agent needs to infer:
|
|
# - "they" refers to Istanbul (from previous message)
|
|
# - User is asking: "Do flights go from Istanbul to Seattle?"
|
|
|
|
async def extract_flight_info_from_messages(messages):
|
|
"""Extract flight information using conversation context."""
|
|
# Build full conversation context
|
|
conversation_context = []
|
|
for msg in messages:
|
|
conversation_context.append({"role": msg.role, "content": msg.content})
|
|
|
|
# Use context-aware extraction prompt
|
|
llm_messages = [
|
|
{"role": "system", "content": FLIGHT_EXTRACTION_PROMPT}
|
|
] + conversation_context[-10:] # Last 10 messages for context
|
|
|
|
response = await plano.chat.completions.create(
|
|
model=FLIGHT_EXTRACTION_MODEL,
|
|
messages=llm_messages,
|
|
temperature=0.1,
|
|
)
|
|
|
|
# LLM extracts: {"origin": "Istanbul", "destination": "Seattle"}
|
|
flight_info = json.loads(response.choices[0].message.content)
|
|
return flight_info
|
|
|
|
**Key Points:**
|
|
|
|
* Include conversation history in extraction prompts
|
|
* Use pronouns and references from context (e.g., "they", "there", "that country")
|
|
* Limit context window to recent messages (e.g., last 10) for efficiency
|
|
* Handle cases where context is insufficient gracefully
|
|
|
|
How Requests Flow
|
|
-----------------
|
|
|
|
Here's how a complete request flows through the Travel Booking system:
|
|
|
|
1. **User sends a prompt**: "What's the weather in London?"
|
|
|
|
2. **Plano analyzes intent**: Plano-Orchestrator analyzes the prompt and determines it should route to ``weather_agent`` based on the agent's description.
|
|
|
|
3. **Plano forwards request**: Plano sends the request to the correct agent with ``/v1/chat/completions``, along with the full conversation history.
|
|
|
|
4. **Weather Agent processes**:
|
|
a. Extracts location "London" from the message
|
|
b. Calls Open-Meteo API to get weather data
|
|
c. Prepares context with weather data
|
|
d. Calls Plano's LLM gateway to generate response
|
|
e. Streams response back to Plano
|
|
|
|
5. **Plano forwards response**: Plano streams the agent's response back to the user.
|
|
|
|
6. **Follow-up question**: User asks "Do they fly out from Seattle?"
|
|
|
|
7. **Plano routes again**: Plano-Orchestrator analyzes the new prompt and routes to ``flight_agent``.
|
|
|
|
8. **Flight Agent processes**:
|
|
a. Extracts flight info using conversation context (infers "they" = London)
|
|
b. Resolves airport codes (London → LHR, Seattle → SEA)
|
|
c. Calls FlightAware API to get flights
|
|
d. Generates response with flight information
|
|
|
|
9. **Seamless handoff**: The user experiences a smooth transition between agents without needing to repeat context.
|
|
|
|
Best Practices
|
|
--------------
|
|
|
|
**Write Clear Agent Descriptions**
|
|
|
|
Agent descriptions are used by Plano-Orchestrator to make routing decisions. Be specific about what each agent handles:
|
|
|
|
.. code-block:: yaml
|
|
|
|
# Good - specific and actionable
|
|
- id: flight_agent
|
|
description: Get live flight information between airports using FlightAware AeroAPI. Shows real-time flight status, scheduled/estimated/actual departure and arrival times, gate and terminal information, delays, aircraft type, and flight status. Automatically resolves city names to airport codes (IATA/ICAO). Understands conversation context to infer origin/destination from follow-up questions.
|
|
|
|
# Less ideal - too vague
|
|
- id: flight_agent
|
|
description: Handles flight queries
|
|
|
|
**Use Conversation Context Effectively**
|
|
|
|
Include conversation history in your extraction and response generation:
|
|
|
|
.. code-block:: python
|
|
|
|
# Include conversation context for extraction
|
|
conversation_context = []
|
|
for msg in messages:
|
|
conversation_context.append({"role": msg.role, "content": msg.content})
|
|
|
|
# Use recent context (last 10 messages)
|
|
context_messages = conversation_context[-10:] if len(conversation_context) > 10 else conversation_context
|
|
|
|
**Route LLM Calls Through Plano's Model Proxy**
|
|
|
|
Always route LLM calls through Plano's :ref:`Model Proxy <llm_providers>` for consistent responses, smart routing, and rich observability:
|
|
|
|
.. code-block:: python
|
|
|
|
plano = AsyncOpenAI(
|
|
base_url=PLANO_ENDPOINT, # Plano's LLM gateway
|
|
api_key="EMPTY",
|
|
)
|
|
|
|
response = await plano.chat.completions.create(
|
|
model="openai/gpt-4o",
|
|
messages=messages,
|
|
stream=True,
|
|
)
|
|
|
|
**Handle Errors Gracefully**
|
|
|
|
Provide fallback values and clear error messages:
|
|
|
|
.. code-block:: python
|
|
|
|
async def extract_location_from_messages(messages):
|
|
try:
|
|
# ... extraction logic ...
|
|
return location
|
|
except Exception as e:
|
|
logger.error(f"Error extracting location: {e}")
|
|
return "New York" # Fallback to default
|
|
|
|
**Use Appropriate Models for Tasks**
|
|
|
|
Use smaller, faster models for extraction tasks and larger models for final responses:
|
|
|
|
.. code-block:: python
|
|
|
|
# Extraction: Use smaller, faster model
|
|
LOCATION_MODEL = "openai/gpt-4o-mini"
|
|
|
|
# Final response: Use larger, more capable model
|
|
WEATHER_MODEL = "openai/gpt-4o"
|
|
|
|
**Stream Responses**
|
|
|
|
Stream responses for better user experience:
|
|
|
|
.. code-block:: python
|
|
|
|
async def stream_chat_completions(request_body):
|
|
response_stream = await plano.chat.completions.create(
|
|
model=WEATHER_MODEL,
|
|
messages=messages,
|
|
stream=True,
|
|
)
|
|
|
|
async for chunk in response_stream:
|
|
if chunk.choices and chunk.choices[0].delta.content:
|
|
yield f"data: {chunk.model_dump_json()}\n\n"
|
|
|
|
Common Use Cases
|
|
----------------
|
|
|
|
Multi-agent orchestration is particularly powerful for:
|
|
|
|
**Travel and Booking Systems**
|
|
|
|
Route queries to specialized agents for weather, flights, hotels, and currency:
|
|
|
|
.. code-block:: yaml
|
|
|
|
agents:
|
|
- id: weather_agent
|
|
description: Get real-time weather conditions and forecasts
|
|
- id: flight_agent
|
|
description: Search for flights and provide flight status
|
|
- id: hotel_agent
|
|
description: Find hotels and check availability
|
|
- id: currency_agent
|
|
description: Provide currency exchange rates
|
|
|
|
**Customer Support**
|
|
|
|
Route common queries to automated support agents while escalating complex issues:
|
|
|
|
.. code-block:: yaml
|
|
|
|
agents:
|
|
- id: tier1_support
|
|
description: Handles common FAQs, password resets, and basic troubleshooting
|
|
- id: tier2_support
|
|
description: Handles complex technical issues requiring deep product knowledge
|
|
- id: human_escalation
|
|
description: Escalates sensitive issues or unresolved problems to human agents
|
|
|
|
**Sales and Marketing**
|
|
|
|
Direct leads and inquiries to specialized sales agents:
|
|
|
|
.. code-block:: yaml
|
|
|
|
agents:
|
|
- id: product_recommendation
|
|
description: Recommends products based on user needs and preferences
|
|
- id: pricing_agent
|
|
description: Provides pricing information and quotes
|
|
- id: sales_closer
|
|
description: Handles final negotiations and closes deals
|
|
|
|
**Technical Documentation and Support**
|
|
|
|
Combine RAG agents for documentation lookup with specialized troubleshooting agents:
|
|
|
|
.. code-block:: yaml
|
|
|
|
agents:
|
|
- id: docs_agent
|
|
description: Retrieves relevant documentation and guides
|
|
filter_chain:
|
|
- query_rewriter
|
|
- context_builder
|
|
- id: troubleshoot_agent
|
|
description: Diagnoses and resolves technical issues step by step
|
|
|
|
Next Steps
|
|
----------
|
|
|
|
* Learn more about :ref:`agents <agents>` and the inner vs. outer loop model
|
|
* Explore :ref:`filter chains <filter_chain>` for adding guardrails and context enrichment
|
|
* See :ref:`observability <observability>` for monitoring multi-agent workflows
|
|
* Review the :ref:`LLM Providers <llm_providers>` guide for model routing within agents
|
|
* Check out the complete `Travel Booking demo <https://github.com/katanemo/plano/tree/main/demos/use_cases/travel_booking>`_ on GitHub
|
|
|
|
.. note::
|
|
To observe traffic to and from agents, please read more about :ref:`observability <observability>` in Plano.
|
|
|
|
By carefully configuring and managing your Agent routing and hand off, you can significantly improve your application's responsiveness, performance, and overall user satisfaction.
|