diff --git a/demos/use_cases/multi_agent_with_crewai_langchain/README.md b/demos/use_cases/multi_agent_with_crewai_langchain/README.md index f9bf9432..3fcc693f 100644 --- a/demos/use_cases/multi_agent_with_crewai_langchain/README.md +++ b/demos/use_cases/multi_agent_with_crewai_langchain/README.md @@ -1,25 +1,132 @@ -# Multi-Framework Travel Agents +Travel Agents in CrewAI and LangChain - with Plano -This demo shows how Plano orchestrates multiple agents built on different frameworks. We run a CrewAI flight agent and a LangChain weather agent side by side to highlight that Plano is framework‑agnostic while still providing a consistent gateway for requests, tools, and telemetry. +**What you'll see:** A travel assistant that seamlessly combines flight booking (CrewAI) and weather forecasts (LangChain) in a single conversation - with unified routing, orchestration, moderation, and observability across both frameworks. -## How it works +## The Problem -Plano sits between clients and agents. Each agent runs independently and exposes its own tools and behavior. The gateway routes requests to the right agent, normalizes requests/responses, and keeps orchestration consistent across frameworks without coupling them together. +Building multi-agent systems today forces developers to: +- **Pick one framework** - can't mix CrewAI, LangChain, or custom agents easily +- **Write plumbing code** - authentication, request routing, error handling +- **Rebuild for changes** - want to swap frameworks? Start over +- **Limited observability** - no unified view across different agent frameworks -## Agents +## Plano's Solution -- **CrewAI Flight Agent** (Port 10520): flight search and itineraries -- **LangChain Weather Agent** (Port 10510): weather forecasts and conditions +Plano acts as a **framework-agnostic proxy and data plane** that: +- Routes requests to the right agent(s), in the right order (CrewAI, LangChain, or custom) +- Normalizes requests/responses across frameworks automatically +- Provides unified authentication, tracing, and logs +- Lets you mix and match frameworks without coupling, so that you can continue to innovate easily -## Quick start +## How To Run + +### Prerequisites + +1. **Install Plano CLI** + ```bash + uv tool install planoai + ``` + +2. **Set Environment Variables** + ```bash + export OPENAI_API_KEY=your_key_here + export AEROAPI_KEY=your_key_here # Get your free API key at https://flightaware.com/aeroapi/ + ``` + +### Start the Demo ```bash -docker compose build -docker compose up -d +# From the demo directory +cd demos/use_cases/multi_agent_with_crewai_langchain + +# Build and start all services +docker-compose up -d ``` -## Environment variables +This starts: +- **Plano** (ports 12000, 8001) - routing and orchestration +- **CrewAI Flight Agent** (port 10520) - flight search +- **LangChain Weather Agent** (port 10510) - weather forecasts +- **Open WebUI** (port 8080) - chat interface +- **Jaeger** (port 16686) - distributed tracing -- `OPENAI_API_KEY`: required for LLM access -- `AEROAPI_KEY`: optional for flight data -- `LLM_GATEWAY_ENDPOINT`: Plano gateway endpoint (default: http://host.docker.internal:12000/v1) +### Try It Out + +1. **Open the Chat Interface** + - Navigate to [http://localhost:8080](http://localhost:8080) + - Create an account (stored locally) + +2. **Ask Multi-Agent Questions** + ``` + "What's the weather in Seattle and can you find flights to San Francisco?" + ``` + + Plano automatically: + - Routes the weather part to the LangChain agent + - Routes the flight part to the CrewAI agent + - Combines responses seamlessly + +3. **View Distributed Traces** + - Open [http://localhost:16686](http://localhost:16686) (Jaeger UI) + - See how requests flow through both agents + + ![Tracing Example](./traces.png) + +## Architecture + +``` +┌─────────────┐ +│ Open WebUI │ (Chat Interface) +└──────┬──────┘ + │ + v +┌─────────────┐ +│ Plano │ (Orchestration & DataPlane) +└──────┬──────┘ + │ + ├──────────────┬──────────────┐ + v v v +┌────────────┐ ┌────────────┐ ┌──────────┐ +│ CrewAI │ │ LangChain │ │ Jaeger │ +│ Flight │ │ Weather │ │ (Traces) │ +│ Agent │ │ Agent │ └──────────┘ +└────────────┘ └────────────┘ + ├──────────────├ + v v +┌─────────────┐ +│ Plano │ (Proxy LLM calls) +└──────┬──────┘ +``` + + +## Travel Agents + +### Flight Agent +- Framework: CrewAI +- Capabilities: Flight search, itinerary planning +- Tools: `resolve_airport_code`, `search_flights` +- Data Source: FlightAware AeroAPI + +### Weather Agent +- Framework: LangChain +- Capabilities: Weather forecasts, conditions +- Tools: `get_weather_forecast` +- Data Source: Open-Meteo API + +## Cleanup + +```bash +docker-compose down +``` + +## Next Steps + +- **Add your own agent** - any framework, just expose the OpenAI-compatible endpoint +- **Custom routing** - modify `config.yaml` to change agent selection logic +- **Production deployment** - see [Plano docs](https://docs.katanemo.com) for scaling guidance + +## Learn More + +- [Plano Documentation](https://docs.planoai.dev) +- [CrewAI Documentation](https://docs.crewai.com) +- [LangChain Documentation](https://python.langchain.com) diff --git a/demos/use_cases/multi_agent_with_crewai_langchain/crewai/flight_agent.py b/demos/use_cases/multi_agent_with_crewai_langchain/crewai/flight_agent.py index d0e8f57c..755da934 100644 --- a/demos/use_cases/multi_agent_with_crewai_langchain/crewai/flight_agent.py +++ b/demos/use_cases/multi_agent_with_crewai_langchain/crewai/flight_agent.py @@ -378,14 +378,14 @@ async def invoke_flight_agent_stream( content = str(chunk) if not content: continue - yield f"data: {json.dumps(create_chat_completion_chunk(model, content))}\n\n" + yield f"data: {create_chat_completion_chunk(model, content).model_dump_json()}\n\n" - yield f"data: {json.dumps(create_chat_completion_chunk(model, '', 'stop'))}\n\n" + yield f"data: {create_chat_completion_chunk(model, '', 'stop').model_dump_json()}\n\n" yield "data: [DONE]\n\n" except Exception as e: logger.error(f"Error streaming response: {e}") error_message = "I'm having trouble retrieving flight information right now. Please try again." - yield f"data: {json.dumps(create_chat_completion_chunk(model, error_message, 'stop'))}\n\n" + yield f"data: {create_chat_completion_chunk(model, error_message, 'stop').model_dump_json()}\n\n" yield "data: [DONE]\n\n" diff --git a/demos/use_cases/multi_agent_with_crewai_langchain/docker-compose.yaml b/demos/use_cases/multi_agent_with_crewai_langchain/docker-compose.yaml index ee1c7b18..851f311e 100644 --- a/demos/use_cases/multi_agent_with_crewai_langchain/docker-compose.yaml +++ b/demos/use_cases/multi_agent_with_crewai_langchain/docker-compose.yaml @@ -1,17 +1,28 @@ + services: plano: - build: - context: ../../../ - dockerfile: Dockerfile + image: katanemo/plano:0.4.2 # Will auto-pull from Docker Hub if not present locally + container_name: plano-dataplane ports: - - "12000:12000" - - "8001:8001" + - "12000:12000" # Main LLM gateway port + - "8001:8001" # Agent/prompt gateway port + - "12001:12001" # Control plane port + - "19901:9901" # Envoy admin interface environment: - - ARCH_CONFIG_PATH=/config/config.yaml + - ARCH_CONFIG_PATH=/app/arch_config.yaml - OPENAI_API_KEY=${OPENAI_API_KEY:?OPENAI_API_KEY environment variable is required but not set} + - OTEL_TRACING_HTTP_ENDPOINT=http://host.docker.internal:4318/v1/traces volumes: - - ./config.yaml:/app/arch_config.yaml + - ./config.yaml:/app/arch_config.yaml:ro - /etc/ssl/cert.pem:/etc/ssl/cert.pem + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:12000/healthz"] + interval: 5s + timeout: 3s + retries: 5 + start_period: 10s + extra_hosts: + - "host.docker.internal:host-gateway" crewai-flight-agent: build: @@ -23,9 +34,12 @@ services: - "10520:10520" environment: - LLM_GATEWAY_ENDPOINT=http://host.docker.internal:12000/v1 - - AEROAPI_KEY=${AEROAPI_KEY:-} + - AEROAPI_KEY=${AEROAPI_KEY:?AEROAPI_KEY environment variable is required but not set} - PYTHONUNBUFFERED=1 command: ["python", "-u", "crewai/flight_agent.py"] + depends_on: + plano: + condition: service_healthy extra_hosts: - "host.docker.internal:host-gateway" @@ -41,6 +55,9 @@ services: - LLM_GATEWAY_ENDPOINT=http://host.docker.internal:12000/v1 - PYTHONUNBUFFERED=1 command: ["python", "-u", "langchain/weather_agent.py"] + depends_on: + plano: + condition: service_healthy extra_hosts: - "host.docker.internal:host-gateway" @@ -54,8 +71,14 @@ services: - ENABLE_OPENAI_API=true - OPENAI_API_BASE_URL=http://host.docker.internal:8001/v1 depends_on: - - langchain-weather-agent - - crewai-flight-agent + plano: + condition: service_healthy + langchain-weather-agent: + condition: service_started + crewai-flight-agent: + condition: service_started + extra_hosts: + - "host.docker.internal:host-gateway" jaeger: build: diff --git a/demos/use_cases/multi_agent_with_crewai_langchain/langchain/weather_agent.py b/demos/use_cases/multi_agent_with_crewai_langchain/langchain/weather_agent.py index c6dafd3a..c977c7aa 100644 --- a/demos/use_cases/multi_agent_with_crewai_langchain/langchain/weather_agent.py +++ b/demos/use_cases/multi_agent_with_crewai_langchain/langchain/weather_agent.py @@ -55,20 +55,20 @@ async def get_weather_data( ): instructions = """You are a city name extractor. Look at the FINAL user message ONLY and extract the city name. -The FINAL user message will be the LAST message with role "user" in the conversation. + The FINAL user message will be the LAST message with role "user" in the conversation. -IMPORTANT: Ignore all previous messages. Focus ONLY on the FINAL user message. + IMPORTANT: Ignore all previous messages. Focus ONLY on the FINAL user message. -Examples of what to extract from the FINAL user message: -- "What's the weather in Seattle?" -> Seattle -- "What's the weather in San Francisco?" -> San Francisco -- "What about Dubai?" -> Dubai -- "How's the weather in Tokyo today?" -> Tokyo -- "Tell me about Lahore" -> Lahore -- "What about there?" -> Look at conversation for the last mentioned city + Examples of what to extract from the FINAL user message: + - "What's the weather in Seattle?" -> Seattle + - "What's the weather in San Francisco?" -> San Francisco + - "What about Dubai?" -> Dubai + - "How's the weather in Tokyo today?" -> Tokyo + - "Tell me about Lahore" -> Lahore + - "What about there?" -> Look at conversation for the last mentioned city -Output ONLY the city name. Nothing else. One word or city name only. -If no city can be found, output: NOT_FOUND""" + Output ONLY the city name. Nothing else. One word or city name only. + If no city can be found, output: NOT_FOUND""" location = city_override if not location: @@ -250,23 +250,24 @@ class WeatherToolInput(BaseModel): ) -WEATHER_SYSTEM_PROMPT = """You are a weather assistant in a multi-agent system. You will receive weather data in JSON format with these fields: +WEATHER_SYSTEM_PROMPT = """ + You are a weather assistant in a multi-agent system. You will receive weather data in JSON format with these fields: -- "location": City name -- "forecast": Array of weather objects, each with date, day_name, temperature_c, temperature_f, temperature_max_c, temperature_min_c, weather_code, sunrise, sunset -- weather_code: WMO code (0=clear, 1-3=partly cloudy, 45-48=fog, 51-67=rain, 71-86=snow, 95-99=thunderstorm) + - "location": City name + - "forecast": Array of weather objects, each with date, day_name, temperature_c, temperature_f, temperature_max_c, temperature_min_c, weather_code, sunrise, sunset + - weather_code: WMO code (0=clear, 1-3=partly cloudy, 45-48=fog, 51-67=rain, 71-86=snow, 95-99=thunderstorm) -Your task: -1. Present the weather/forecast clearly for the location -2. For single day: show current conditions -3. For multi-day: show each day with date and conditions -4. Include temperature in both Celsius and Fahrenheit -5. Describe conditions naturally based on weather_code -6. Use conversational language + Your task: + 1. Present the weather/forecast clearly for the location + 2. For single day: show current conditions + 3. For multi-day: show each day with date and conditions + 4. Include temperature in both Celsius and Fahrenheit + 5. Describe conditions naturally based on weather_code + 6. Use conversational language -Multi-agent context: You are part of a larger system. If the conversation includes additional context or information from other sources, acknowledge and incorporate it naturally into your response. Your primary focus is weather, but be aware of the full conversation context. + Multi-agent context: You are part of a larger system. If the conversation includes additional context or information from other sources, acknowledge and incorporate it naturally into your response. Your primary focus is weather, but be aware of the full conversation context. -Remember: Only use the provided data. If fields are null, mention data is unavailable.""" + Remember: Only use the provided data. If fields are null, mention data is unavailable.""" def build_weather_agent( @@ -405,20 +406,20 @@ async def invoke_weather_agent_stream( ).strip() if not content: continue - yield f"data: {json.dumps(create_chat_completion_chunk(model, content))}\n\n" + yield f"data: {create_chat_completion_chunk(model, content).model_dump_json()}\n\n" - yield f"data: {json.dumps(create_chat_completion_chunk(model, '', 'stop'))}\n\n" + yield f"data: {create_chat_completion_chunk(model, '', 'stop').model_dump_json()}\n\n" yield "data: [DONE]\n\n" except Exception as e: logger.error("Error streaming weather response: %s", e) error_message = "I'm having trouble retrieving weather information right now. Please try again." - yield f"data: {json.dumps(create_chat_completion_chunk(model, error_message, 'stop'))}\n\n" + yield f"data: {create_chat_completion_chunk(model, error_message, 'stop').model_dump_json()}\n\n" yield "data: [DONE]\n\n" async def invoke_weather_agent_error_stream(request_body: dict, error_message: str): model = request_body.get("model", WEATHER_MODEL) - yield f"data: {json.dumps(create_chat_completion_chunk(model, error_message, 'stop'))}\n\n" + yield f"data: {create_chat_completion_chunk(model, error_message, 'stop').model_dump_json()}\n\n" yield "data: [DONE]\n\n" diff --git a/demos/use_cases/multi_agent_with_crewai_langchain/openai_protocol.py b/demos/use_cases/multi_agent_with_crewai_langchain/openai_protocol.py index 1c8000a2..c50574de 100644 --- a/demos/use_cases/multi_agent_with_crewai_langchain/openai_protocol.py +++ b/demos/use_cases/multi_agent_with_crewai_langchain/openai_protocol.py @@ -1,15 +1,16 @@ """OpenAI API protocol utilities for standardized response formatting.""" import time -import uuid from typing import Optional +from openai.types.chat import ChatCompletionChunk +from openai.types.chat.chat_completion_chunk import Choice, ChoiceDelta def create_chat_completion_chunk( model: str, content: str, finish_reason: Optional[str] = None, -) -> dict: +) -> ChatCompletionChunk: """Create an OpenAI-compatible streaming chat completion chunk. Args: @@ -18,18 +19,18 @@ def create_chat_completion_chunk( finish_reason: Optional finish reason ('stop', 'length', etc.) Returns: - Dictionary formatted as OpenAI chat.completion.chunk + ChatCompletionChunk object from OpenAI SDK """ - return { - "id": f"chatcmpl-{uuid.uuid4().hex[:8]}", - "object": "chat.completion.chunk", - "created": int(time.time()), - "model": model, - "choices": [ - { - "index": 0, - "delta": {"content": content} if content else {}, - "finish_reason": finish_reason, - } + return ChatCompletionChunk( + id=f"chatcmpl-{int(time.time() * 1000000)}", + object="chat.completion.chunk", + created=int(time.time()), + model=model, + choices=[ + Choice( + index=0, + delta=ChoiceDelta(content=content) if content else ChoiceDelta(), + finish_reason=finish_reason, + ) ], - } + ) diff --git a/demos/use_cases/multi_agent_with_crewai_langchain/traces.png b/demos/use_cases/multi_agent_with_crewai_langchain/traces.png new file mode 100644 index 00000000..5fd2bd14 Binary files /dev/null and b/demos/use_cases/multi_agent_with_crewai_langchain/traces.png differ