mirror of
https://github.com/katanemo/plano.git
synced 2026-06-17 15:25:17 +02:00
feat: routing configuration with external policy provider support
This commit is contained in:
parent
5aeb69e034
commit
2c643532ef
4 changed files with 123 additions and 0 deletions
|
|
@ -193,6 +193,65 @@ Clients can let the router decide or still specify aliases:
|
|||
# No model specified - router will analyze and choose claude-sonnet-4-5
|
||||
)
|
||||
|
||||
External Policy Provider (policy_id)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
For multitenant deployments, Plano can fetch routing preferences from an external HTTP endpoint using a ``policy_id`` provided by the caller.
|
||||
|
||||
Resolution order is:
|
||||
|
||||
1. Inline ``routing_policy`` in request payload
|
||||
2. ``policy_id`` lookup via ``routing.policy_provider``
|
||||
3. Metadata ``plano_preference_config``
|
||||
4. Config-file ``routing_preferences``
|
||||
|
||||
.. code-block:: yaml
|
||||
:caption: External Policy Provider Configuration
|
||||
|
||||
routing:
|
||||
model_provider: arch-router
|
||||
model: Arch-Router
|
||||
policy_provider:
|
||||
url: https://my-service.internal/v1/routing-policy
|
||||
headers:
|
||||
Authorization: Bearer $POLICY_API_KEY
|
||||
ttl_seconds: 300
|
||||
|
||||
When ``policy_id`` is provided and no inline ``routing_policy`` is present, Plano fetches:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
GET https://my-service.internal/v1/routing-policy?policy_id=customer-abc-123
|
||||
|
||||
.. code-block:: json
|
||||
:caption: Routing request with policy_id
|
||||
|
||||
{
|
||||
"messages": [{"role": "user", "content": "Help me summarize this"}],
|
||||
"policy_id": "customer-abc-123"
|
||||
}
|
||||
|
||||
.. code-block:: json
|
||||
:caption: Expected response from external policy endpoint
|
||||
|
||||
{
|
||||
"policy_id": "customer-abc-123",
|
||||
"routing_preferences": [
|
||||
{
|
||||
"model": "openai/gpt-4o",
|
||||
"routing_preferences": [
|
||||
{"name": "quick response", "description": "fast lightweight responses"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"model": "anthropic/claude-sonnet-4-0",
|
||||
"routing_preferences": [
|
||||
{"name": "deep analysis", "description": "comprehensive detailed analysis"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
Arch-Router
|
||||
-----------
|
||||
|
|
|
|||
|
|
@ -47,6 +47,18 @@ model_aliases:
|
|||
smart-llm:
|
||||
target: gpt-4o
|
||||
|
||||
# Optional routing policy provider for multitenant preference-based routing.
|
||||
# If policy_id is included in the request and inline routing_policy is absent,
|
||||
# Plano fetches routing preferences from this endpoint and caches by policy_id.
|
||||
routing:
|
||||
model_provider: arch-router
|
||||
model: Arch-Router
|
||||
policy_provider:
|
||||
url: https://my-service.internal/v1/routing-policy
|
||||
headers:
|
||||
Authorization: Bearer $POLICY_API_KEY
|
||||
ttl_seconds: 300
|
||||
|
||||
# HTTP listeners - entry points for agent routing, prompt targets, and direct LLM access
|
||||
listeners:
|
||||
# Agent listener for routing requests to multiple agents
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue