using docker-compose to use the planoai images and updated agents based on PR feedback

demos/use_cases/multi_agent_with_crewai_langchain/README.md

@@ -1,25 +1,132 @@
-# Multi-Framework Travel Agents
+Travel Agents in CrewAI and LangChain - with Plano
 
-This demo shows how Plano orchestrates multiple agents built on different frameworks. We run a CrewAI flight agent and a LangChain weather agent side by side to highlight that Plano is framework‑agnostic while still providing a consistent gateway for requests, tools, and telemetry.
+**What you'll see:** A travel assistant that seamlessly combines flight booking (CrewAI) and weather forecasts (LangChain) in a single conversation - with unified routing, orchestration, moderation, and observability across both frameworks.
 
-## How it works
+## The Problem
 
-Plano sits between clients and agents. Each agent runs independently and exposes its own tools and behavior. The gateway routes requests to the right agent, normalizes requests/responses, and keeps orchestration consistent across frameworks without coupling them together.
+Building multi-agent systems today forces developers to:
+- **Pick one framework** - can't mix CrewAI, LangChain, or custom agents easily
+- **Write plumbing code** - authentication, request routing, error handling
+- **Rebuild for changes** - want to swap frameworks? Start over
+- **Limited observability** - no unified view across different agent frameworks
 
-## Agents
+## Plano's Solution
 
-- **CrewAI Flight Agent** (Port 10520): flight search and itineraries
-- **LangChain Weather Agent** (Port 10510): weather forecasts and conditions
+Plano acts as a **framework-agnostic proxy and data plane** that:
+- Routes requests to the right agent(s), in the right order (CrewAI, LangChain, or custom)
+- Normalizes requests/responses across frameworks automatically
+- Provides unified authentication, tracing, and logs
+- Lets you mix and match frameworks without coupling, so that you can continue to innovate easily
 
-## Quick start
+## How To Run
+
+### Prerequisites
+
+1. **Install Plano CLI**
+   ```bash
+   uv tool install planoai
+   ```
+
+2. **Set Environment Variables**
+   ```bash
+   export OPENAI_API_KEY=your_key_here
+   export AEROAPI_KEY=your_key_here  # Get your free API key at https://flightaware.com/aeroapi/
+   ```
+
+### Start the Demo
 
 ```bash
-docker compose build
-docker compose up -d
+# From the demo directory
+cd demos/use_cases/multi_agent_with_crewai_langchain
+
+# Build and start all services
+docker-compose up -d
 ```
 
-## Environment variables
+This starts:
+- **Plano** (ports 12000, 8001) - routing and orchestration
+- **CrewAI Flight Agent** (port 10520) - flight search
+- **LangChain Weather Agent** (port 10510) - weather forecasts
+- **Open WebUI** (port 8080) - chat interface
+- **Jaeger** (port 16686) - distributed tracing
 
-- `OPENAI_API_KEY`: required for LLM access
-- `AEROAPI_KEY`: optional for flight data
-- `LLM_GATEWAY_ENDPOINT`: Plano gateway endpoint (default: http://host.docker.internal:12000/v1)
+### Try It Out
+
+1. **Open the Chat Interface**
+   - Navigate to [http://localhost:8080](http://localhost:8080)
+   - Create an account (stored locally)
+
+2. **Ask Multi-Agent Questions**
+   ```
+   "What's the weather in Seattle and can you find flights to San Francisco?"
+   ```
+
+   Plano automatically:
+   - Routes the weather part to the LangChain agent
+   - Routes the flight part to the CrewAI agent
+   - Combines responses seamlessly
+
+3. **View Distributed Traces**
+   - Open [http://localhost:16686](http://localhost:16686) (Jaeger UI)
+   - See how requests flow through both agents
+
+   ![Tracing Example](./traces.png)
+
+## Architecture
+
+```
+┌─────────────┐
+│  Open WebUI │ (Chat Interface)
+└──────┬──────┘
+       │
+       v
+┌─────────────┐
+│    Plano    │ (Orchestration & DataPlane)
+└──────┬──────┘
+       │
+       ├──────────────┬──────────────┐
+       v              v              v
+┌────────────┐ ┌────────────┐ ┌──────────┐
+│  CrewAI    │ │ LangChain  │ │  Jaeger  │
+│   Flight   │ │  Weather   │ │ (Traces) │
+│   Agent    │ │   Agent    │ └──────────┘
+└────────────┘ └────────────┘
+       ├──────────────├
+       v              v
+┌─────────────┐
+│    Plano    │ (Proxy LLM calls)
+└──────┬──────┘
+```
+
+
+## Travel Agents
+
+### Flight Agent
+- Framework: CrewAI
+- Capabilities: Flight search, itinerary planning
+- Tools: `resolve_airport_code`, `search_flights`
+- Data Source: FlightAware AeroAPI
+
+### Weather Agent
+- Framework: LangChain
+- Capabilities: Weather forecasts, conditions
+- Tools: `get_weather_forecast`
+- Data Source: Open-Meteo API
+
+## Cleanup
+
+```bash
+docker-compose down
+```
+
+## Next Steps
+
+- **Add your own agent** - any framework, just expose the OpenAI-compatible endpoint
+- **Custom routing** - modify `config.yaml` to change agent selection logic
+- **Production deployment** - see [Plano docs](https://docs.katanemo.com) for scaling guidance
+
+## Learn More
+
+- [Plano Documentation](https://docs.planoai.dev)
+- [CrewAI Documentation](https://docs.crewai.com)
+- [LangChain Documentation](https://python.langchain.com)

demos/use_cases/multi_agent_with_crewai_langchain/crewai/flight_agent.py

@@ -378,14 +378,14 @@ async def invoke_flight_agent_stream(
                 content = str(chunk)
             if not content:
                 continue
-            yield f"data: {json.dumps(create_chat_completion_chunk(model, content))}\n\n"
+            yield f"data: {create_chat_completion_chunk(model, content).model_dump_json()}\n\n"
 
-        yield f"data: {json.dumps(create_chat_completion_chunk(model, '', 'stop'))}\n\n"
+        yield f"data: {create_chat_completion_chunk(model, '', 'stop').model_dump_json()}\n\n"
         yield "data: [DONE]\n\n"
     except Exception as e:
         logger.error(f"Error streaming response: {e}")
         error_message = "I'm having trouble retrieving flight information right now. Please try again."
-        yield f"data: {json.dumps(create_chat_completion_chunk(model, error_message, 'stop'))}\n\n"
+        yield f"data: {create_chat_completion_chunk(model, error_message, 'stop').model_dump_json()}\n\n"
         yield "data: [DONE]\n\n"
 
 

demos/use_cases/multi_agent_with_crewai_langchain/docker-compose.yaml

@@ -1,17 +1,28 @@
+
 services:
   plano:
-    build:
-      context: ../../../
-      dockerfile: Dockerfile
+    image: katanemo/plano:0.4.2  # Will auto-pull from Docker Hub if not present locally
+    container_name: plano-dataplane
     ports:
-      - "12000:12000"
-      - "8001:8001"
+      - "12000:12000"  # Main LLM gateway port
+      - "8001:8001"    # Agent/prompt gateway port
+      - "12001:12001"  # Control plane port
+      - "19901:9901"   # Envoy admin interface
     environment:
-      - ARCH_CONFIG_PATH=/config/config.yaml
+      - ARCH_CONFIG_PATH=/app/arch_config.yaml
       - OPENAI_API_KEY=${OPENAI_API_KEY:?OPENAI_API_KEY environment variable is required but not set}
+      - OTEL_TRACING_HTTP_ENDPOINT=http://host.docker.internal:4318/v1/traces
     volumes:
-      - ./config.yaml:/app/arch_config.yaml
+      - ./config.yaml:/app/arch_config.yaml:ro
       - /etc/ssl/cert.pem:/etc/ssl/cert.pem
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:12000/healthz"]
+      interval: 5s
+      timeout: 3s
+      retries: 5
+      start_period: 10s
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
 
   crewai-flight-agent:
     build:
@@ -23,9 +34,12 @@ services:
       - "10520:10520"
     environment:
       - LLM_GATEWAY_ENDPOINT=http://host.docker.internal:12000/v1
-      - AEROAPI_KEY=${AEROAPI_KEY:-}
+      - AEROAPI_KEY=${AEROAPI_KEY:?AEROAPI_KEY environment variable is required but not set}
       - PYTHONUNBUFFERED=1
     command: ["python", "-u", "crewai/flight_agent.py"]
+    depends_on:
+      plano:
+        condition: service_healthy
     extra_hosts:
       - "host.docker.internal:host-gateway"
 
@@ -41,6 +55,9 @@ services:
       - LLM_GATEWAY_ENDPOINT=http://host.docker.internal:12000/v1
       - PYTHONUNBUFFERED=1
     command: ["python", "-u", "langchain/weather_agent.py"]
+    depends_on:
+      plano:
+        condition: service_healthy
     extra_hosts:
       - "host.docker.internal:host-gateway"
 
@@ -54,8 +71,14 @@ services:
       - ENABLE_OPENAI_API=true
       - OPENAI_API_BASE_URL=http://host.docker.internal:8001/v1
     depends_on:
-      - langchain-weather-agent
-      - crewai-flight-agent
+      plano:
+        condition: service_healthy
+      langchain-weather-agent:
+        condition: service_started
+      crewai-flight-agent:
+        condition: service_started
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
 
   jaeger:
     build:

demos/use_cases/multi_agent_with_crewai_langchain/langchain/weather_agent.py

@@ -55,20 +55,20 @@ async def get_weather_data(
 ):
     instructions = """You are a city name extractor. Look at the FINAL user message ONLY and extract the city name.
 
-The FINAL user message will be the LAST message with role "user" in the conversation.
+    The FINAL user message will be the LAST message with role "user" in the conversation.
 
-IMPORTANT: Ignore all previous messages. Focus ONLY on the FINAL user message.
+    IMPORTANT: Ignore all previous messages. Focus ONLY on the FINAL user message.
 
-Examples of what to extract from the FINAL user message:
-- "What's the weather in Seattle?" -> Seattle
-- "What's the weather in San Francisco?" -> San Francisco
-- "What about Dubai?" -> Dubai
-- "How's the weather in Tokyo today?" -> Tokyo
-- "Tell me about Lahore" -> Lahore
-- "What about there?" -> Look at conversation for the last mentioned city
+    Examples of what to extract from the FINAL user message:
+    - "What's the weather in Seattle?" -> Seattle
+    - "What's the weather in San Francisco?" -> San Francisco
+    - "What about Dubai?" -> Dubai
+    - "How's the weather in Tokyo today?" -> Tokyo
+    - "Tell me about Lahore" -> Lahore
+    - "What about there?" -> Look at conversation for the last mentioned city
 
-Output ONLY the city name. Nothing else. One word or city name only.
-If no city can be found, output: NOT_FOUND"""
+    Output ONLY the city name. Nothing else. One word or city name only.
+    If no city can be found, output: NOT_FOUND"""
 
     location = city_override
     if not location:
@@ -250,23 +250,24 @@ class WeatherToolInput(BaseModel):
     )
 
 
-WEATHER_SYSTEM_PROMPT = """You are a weather assistant in a multi-agent system. You will receive weather data in JSON format with these fields:
+WEATHER_SYSTEM_PROMPT = """
+    You are a weather assistant in a multi-agent system. You will receive weather data in JSON format with these fields:
 
-- "location": City name
-- "forecast": Array of weather objects, each with date, day_name, temperature_c, temperature_f, temperature_max_c, temperature_min_c, weather_code, sunrise, sunset
-- weather_code: WMO code (0=clear, 1-3=partly cloudy, 45-48=fog, 51-67=rain, 71-86=snow, 95-99=thunderstorm)
+    - "location": City name
+    - "forecast": Array of weather objects, each with date, day_name, temperature_c, temperature_f, temperature_max_c, temperature_min_c, weather_code, sunrise, sunset
+    - weather_code: WMO code (0=clear, 1-3=partly cloudy, 45-48=fog, 51-67=rain, 71-86=snow, 95-99=thunderstorm)
 
-Your task:
-1. Present the weather/forecast clearly for the location
-2. For single day: show current conditions
-3. For multi-day: show each day with date and conditions
-4. Include temperature in both Celsius and Fahrenheit
-5. Describe conditions naturally based on weather_code
-6. Use conversational language
+    Your task:
+    1. Present the weather/forecast clearly for the location
+    2. For single day: show current conditions
+    3. For multi-day: show each day with date and conditions
+    4. Include temperature in both Celsius and Fahrenheit
+    5. Describe conditions naturally based on weather_code
+    6. Use conversational language
 
-Multi-agent context: You are part of a larger system. If the conversation includes additional context or information from other sources, acknowledge and incorporate it naturally into your response. Your primary focus is weather, but be aware of the full conversation context.
+    Multi-agent context: You are part of a larger system. If the conversation includes additional context or information from other sources, acknowledge and incorporate it naturally into your response. Your primary focus is weather, but be aware of the full conversation context.
 
-Remember: Only use the provided data. If fields are null, mention data is unavailable."""
+    Remember: Only use the provided data. If fields are null, mention data is unavailable."""
 
 
 def build_weather_agent(
@@ -405,20 +406,20 @@ async def invoke_weather_agent_stream(
                 ).strip()
                 if not content:
                     continue
-            yield f"data: {json.dumps(create_chat_completion_chunk(model, content))}\n\n"
+            yield f"data: {create_chat_completion_chunk(model, content).model_dump_json()}\n\n"
 
-        yield f"data: {json.dumps(create_chat_completion_chunk(model, '', 'stop'))}\n\n"
+        yield f"data: {create_chat_completion_chunk(model, '', 'stop').model_dump_json()}\n\n"
         yield "data: [DONE]\n\n"
     except Exception as e:
         logger.error("Error streaming weather response: %s", e)
         error_message = "I'm having trouble retrieving weather information right now. Please try again."
-        yield f"data: {json.dumps(create_chat_completion_chunk(model, error_message, 'stop'))}\n\n"
+        yield f"data: {create_chat_completion_chunk(model, error_message, 'stop').model_dump_json()}\n\n"
         yield "data: [DONE]\n\n"
 
 
 async def invoke_weather_agent_error_stream(request_body: dict, error_message: str):
     model = request_body.get("model", WEATHER_MODEL)
-    yield f"data: {json.dumps(create_chat_completion_chunk(model, error_message, 'stop'))}\n\n"
+    yield f"data: {create_chat_completion_chunk(model, error_message, 'stop').model_dump_json()}\n\n"
     yield "data: [DONE]\n\n"
 
 

demos/use_cases/multi_agent_with_crewai_langchain/openai_protocol.py

@@ -1,15 +1,16 @@
 """OpenAI API protocol utilities for standardized response formatting."""
 
 import time
-import uuid
 from typing import Optional
+from openai.types.chat import ChatCompletionChunk
+from openai.types.chat.chat_completion_chunk import Choice, ChoiceDelta
 
 
 def create_chat_completion_chunk(
     model: str,
     content: str,
     finish_reason: Optional[str] = None,
-) -> dict:
+) -> ChatCompletionChunk:
     """Create an OpenAI-compatible streaming chat completion chunk.
 
     Args:
@@ -18,18 +19,18 @@ def create_chat_completion_chunk(
         finish_reason: Optional finish reason ('stop', 'length', etc.)
 
     Returns:
-        Dictionary formatted as OpenAI chat.completion.chunk
+        ChatCompletionChunk object from OpenAI SDK
     """
-    return {
-        "id": f"chatcmpl-{uuid.uuid4().hex[:8]}",
-        "object": "chat.completion.chunk",
-        "created": int(time.time()),
-        "model": model,
-        "choices": [
-            {
-                "index": 0,
-                "delta": {"content": content} if content else {},
-                "finish_reason": finish_reason,
-            }
+    return ChatCompletionChunk(
+        id=f"chatcmpl-{int(time.time() * 1000000)}",
+        object="chat.completion.chunk",
+        created=int(time.time()),
+        model=model,
+        choices=[
+            Choice(
+                index=0,
+                delta=ChoiceDelta(content=content) if content else ChoiceDelta(),
+                finish_reason=finish_reason,
+            )
         ],
-    }
+    )