When deploying Plano in front of LLM proxy services that manage their own API key validation (such as LiteLLM, OpenRouter, or custom gateways), you may want to forward the client’s original Authorization header instead of replacing it with a configured access_key.
The passthrough_auth option enables this behavior:
llm_providers:
+ # Forward client's Authorization header to LiteLLM
+ - model: openai/gpt-4o-litellm
+ base_url: https://litellm.example.com
+ passthrough_auth: true
+ default: true
+
+ # Forward to OpenRouter
+ - model: openai/claude-3-opus
+ base_url: https://openrouter.ai/api/v1
+ passthrough_auth: true
+How it works:
+Client sends a request with Authorization: Bearer <virtual-key>
Plano preserves this header instead of replacing it with access_key
The upstream service (e.g., LiteLLM) validates the virtual key
Response flows back through Plano to the client
Use Cases:
+LiteLLM Integration: Route requests to LiteLLM which manages virtual keys and rate limits
OpenRouter: Forward requests to OpenRouter with per-user API keys
Custom API Gateways: Integrate with internal gateways that have their own authentication
Multi-tenant Deployments: Allow different clients to use their own credentials
Important Notes:
+When passthrough_auth: true is set, the access_key field is ignored (a warning is logged if both are configured)
If the client doesn’t provide an Authorization header, the request is forwarded without authentication (upstream will likely return 401)
The base_url is typically required when using passthrough_auth
Configuration with LiteLLM example:
+# plano_config.yaml
+version: v0.3.0
+
+listeners:
+ - name: llm
+ type: model
+ port: 10000
+
+model_providers:
+ - model: openai/gpt-4o
+ base_url: https://litellm.example.com
+ passthrough_auth: true
+ default: true
+# Client request - virtual key is forwarded to upstream
+curl http://localhost:10000/v1/chat/completions \
+ -H "Authorization: Bearer sk-litellm-virtual-key-abc123" \
+ -H "Content-Type: application/json" \
+ -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'
+ 1# Arch Gateway configuration version
2version: v0.3.0
3
- 4
- 5# External HTTP agents - API type is controlled by request path (/v1/responses, /v1/messages, /v1/chat/completions)
- 6agents:
- 7 - id: weather_agent # Example agent for weather
- 8 url: http://host.docker.internal:10510
- 9
- 10 - id: flight_agent # Example agent for flights
- 11 url: http://host.docker.internal:10520
- 12
- 13
- 14# MCP filters applied to requests/responses (e.g., input validation, query rewriting)
- 15filters:
- 16 - id: input_guards # Example filter for input validation
- 17 url: http://host.docker.internal:10500
- 18 # type: mcp (default)
- 19 # transport: streamable-http (default)
- 20 # tool: input_guards (default - same as filter id)
- 21
- 22
- 23# LLM provider configurations with API keys and model routing
- 24model_providers:
- 25 - model: openai/gpt-4o
- 26 access_key: $OPENAI_API_KEY
- 27 default: true
+ 4# External HTTP agents - API type is controlled by request path (/v1/responses, /v1/messages, /v1/chat/completions)
+ 5agents:
+ 6 - id: weather_agent # Example agent for weather
+ 7 url: http://host.docker.internal:10510
+ 8
+ 9 - id: flight_agent # Example agent for flights
+ 10 url: http://host.docker.internal:10520
+ 11
+ 12# MCP filters applied to requests/responses (e.g., input validation, query rewriting)
+ 13filters:
+ 14 - id: input_guards # Example filter for input validation
+ 15 url: http://host.docker.internal:10500
+ 16 # type: mcp (default)
+ 17 # transport: streamable-http (default)
+ 18 # tool: input_guards (default - same as filter id)
+ 19
+ 20# LLM provider configurations with API keys and model routing
+ 21model_providers:
+ 22 - model: openai/gpt-4o
+ 23 access_key: $OPENAI_API_KEY
+ 24 default: true
+ 25
+ 26 - model: openai/gpt-4o-mini
+ 27 access_key: $OPENAI_API_KEY
28
- 29 - model: openai/gpt-4o-mini
- 30 access_key: $OPENAI_API_KEY
+ 29 - model: anthropic/claude-sonnet-4-0
+ 30 access_key: $ANTHROPIC_API_KEY
31
- 32 - model: anthropic/claude-sonnet-4-0
- 33 access_key: $ANTHROPIC_API_KEY
+ 32 - model: mistral/ministral-3b-latest
+ 33 access_key: $MISTRAL_API_KEY
34
- 35 - model: mistral/ministral-3b-latest
- 36 access_key: $MISTRAL_API_KEY
- 37
- 38
- 39# Model aliases - use friendly names instead of full provider model names
- 40model_aliases:
- 41 fast-llm:
- 42 target: gpt-4o-mini
- 43
- 44 smart-llm:
- 45 target: gpt-4o
+ 35 # Example: Passthrough authentication for LiteLLM or similar proxies
+ 36 # When passthrough_auth is true, client's Authorization header is forwarded
+ 37 # instead of using the configured access_key
+ 38 - model: openai/gpt-4o-litellm
+ 39 base_url: https://litellm.example.com
+ 40 passthrough_auth: true
+ 41
+ 42# Model aliases - use friendly names instead of full provider model names
+ 43model_aliases:
+ 44 fast-llm:
+ 45 target: gpt-4o-mini
46
- 47
- 48# HTTP listeners - entry points for agent routing, prompt targets, and direct LLM access
- 49listeners:
- 50 # Agent listener for routing requests to multiple agents
- 51 - type: agent
- 52 name: travel_booking_service
- 53 port: 8001
- 54 router: plano_orchestrator_v1
- 55 address: 0.0.0.0
- 56 agents:
- 57 - id: rag_agent
- 58 description: virtual assistant for retrieval augmented generation tasks
- 59 filter_chain:
- 60 - input_guards
- 61
- 62 # Model listener for direct LLM access
- 63 - type: model
- 64 name: model_1
- 65 address: 0.0.0.0
- 66 port: 12000
- 67
- 68 # Prompt listener for function calling (for prompt_targets)
- 69 - type: prompt
- 70 name: prompt_function_listener
- 71 address: 0.0.0.0
- 72 port: 10000
- 73 # This listener is used for prompt_targets and function calling
- 74
- 75
- 76# Reusable service endpoints
- 77endpoints:
- 78 app_server:
- 79 endpoint: 127.0.0.1:80
- 80 connect_timeout: 0.005s
- 81
- 82 mistral_local:
- 83 endpoint: 127.0.0.1:8001
- 84
+ 47 smart-llm:
+ 48 target: gpt-4o
+ 49
+ 50# HTTP listeners - entry points for agent routing, prompt targets, and direct LLM access
+ 51listeners:
+ 52 # Agent listener for routing requests to multiple agents
+ 53 - type: agent
+ 54 name: travel_booking_service
+ 55 port: 8001
+ 56 router: plano_orchestrator_v1
+ 57 address: 0.0.0.0
+ 58 agents:
+ 59 - id: rag_agent
+ 60 description: virtual assistant for retrieval augmented generation tasks
+ 61 filter_chain:
+ 62 - input_guards
+ 63
+ 64 # Model listener for direct LLM access
+ 65 - type: model
+ 66 name: model_1
+ 67 address: 0.0.0.0
+ 68 port: 12000
+ 69
+ 70 # Prompt listener for function calling (for prompt_targets)
+ 71 - type: prompt
+ 72 name: prompt_function_listener
+ 73 address: 0.0.0.0
+ 74 port: 10000
+ 75 # This listener is used for prompt_targets and function calling
+ 76
+ 77# Reusable service endpoints
+ 78endpoints:
+ 79 app_server:
+ 80 endpoint: 127.0.0.1:80
+ 81 connect_timeout: 0.005s
+ 82
+ 83 mistral_local:
+ 84 endpoint: 127.0.0.1:8001
85
86# Prompt targets for function calling and API orchestration
87prompt_targets:
@@ -268,11 +268,10 @@ where prompts get routed to, apply guardrails, and enable critical agent observa
102 path: /weather
103 http_method: POST
104
-105
-106# OpenTelemetry tracing configuration
-107tracing:
-108 # Random sampling percentage (1-100)
-109 random_sampling: 100
+105# OpenTelemetry tracing configuration
+106tracing:
+107 # Random sampling percentage (1-100)
+108 random_sampling: 100