apunkt/plano
1
0
Fork 0
mirror of https://github.com/katanemo/plano.git synced 2026-06-17 15:25:17 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat(docs): refresh routing models

Browse source
This commit is contained in:
Musa 2025-12-23 14:02:20 -08:00
parent 9c515116cb
commit ec17252403
3 changed files with 31 additions and 30 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs/source/concepts/llm_providers/supported_providers.rst
View file

@ -17,6 +17,7 @@ Configuration Structure
All providers are configured in the ``llm_providers`` section of your ``plano_config.yaml`` file:
.. code-block:: yaml
llm_providers:
# Provider configurations go here
- model: provider/model-name

46
docs/source/guides/llm_router.rst
View file

@ -18,7 +18,7 @@ Model-based routing
Direct routing allows you to specify exact provider and model combinations using the format ``provider/model-name``:
- Use provider-specific names like ``openai/gpt-4o`` or ``anthropic/claude-3-5-sonnet-20241022``
- Use provider-specific names like ``openai/gpt-5.2`` or ``anthropic/claude-sonnet-4-5``
- Provides full control and transparency over which model handles each request
- Ideal for production workloads where you want predictable routing behavior
@ -38,14 +38,14 @@ Configure your LLM providers with specific provider/model names:
timeout: 30s
llm_providers:
- model: openai/gpt-4o-mini
- model: openai/gpt-5.2
access_key: $OPENAI_API_KEY
default: true
- model: openai/gpt-4o
- model: openai/gpt-5
access_key: $OPENAI_API_KEY
- model: anthropic/claude-3-5-sonnet-20241022
- model: anthropic/claude-sonnet-4-5
access_key: $ANTHROPIC_API_KEY
Client usage
@ -57,12 +57,12 @@ Clients specify exact models:
# Direct provider/model specification
response = client.chat.completions.create(
model="openai/gpt-4o-mini",
model="openai/gpt-5.2",
messages=[{"role": "user", "content": "Hello!"}]
)
response = client.chat.completions.create(
model="anthropic/claude-3-5-sonnet-20241022",
model="anthropic/claude-sonnet-4-5",
messages=[{"role": "user", "content": "Write a story"}]
)
@ -73,7 +73,7 @@ Alias-based routing
Alias-based routing lets you create semantic model names that decouple your application from specific providers:
- Use meaningful names like ``fast-model``, ``reasoning-model``, or ``arch.summarize.v1`` (see :ref:`model_aliases`)
- Use meaningful names like ``fast-model``, ``reasoning-model``, or ``plano.summarize.v1`` (see :ref:`model_aliases`)
- Maps semantic names to underlying provider models for easier experimentation and provider switching
- Ideal for applications that want abstraction from specific model names while maintaining control
@ -93,25 +93,25 @@ Configure semantic aliases that map to underlying models:
timeout: 30s
llm_providers:
- model: openai/gpt-4o-mini
- model: openai/gpt-5.2
access_key: $OPENAI_API_KEY
- model: openai/gpt-4o
- model: openai/gpt-5
access_key: $OPENAI_API_KEY
- model: anthropic/claude-3-5-sonnet-20241022
- model: anthropic/claude-sonnet-4-5
access_key: $ANTHROPIC_API_KEY
model_aliases:
# Model aliases - friendly names that map to actual provider names
fast-model:
target: gpt-4o-mini
target: gpt-5.2
reasoning-model:
target: gpt-4o
target: gpt-5
creative-model:
target: claude-3-5-sonnet-20241022
target: claude-sonnet-4-5
Client usage
^^^^^^^^^^^^
@ -160,11 +160,11 @@ To configure preference-aligned dynamic routing, define routing preferences that
timeout: 30s
llm_providers:
- model: openai/gpt-4o-mini
- model: openai/gpt-5.2
access_key: $OPENAI_API_KEY
default: true
- model: openai/gpt-4o
- model: openai/gpt-5
access_key: $OPENAI_API_KEY
routing_preferences:
- name: code understanding
@ -172,7 +172,7 @@ To configure preference-aligned dynamic routing, define routing preferences that
- name: complex reasoning
description: deep analysis, mathematical problem solving, and logical reasoning
- model: anthropic/claude-3-5-sonnet-20241022
- model: anthropic/claude-sonnet-4-5
access_key: $ANTHROPIC_API_KEY
routing_preferences:
- name: creative writing
@ -190,7 +190,7 @@ Clients can let the router decide or still specify aliases:
# Let Arch-Router choose based on content
response = client.chat.completions.create(
messages=[{"role": "user", "content": "Write a creative story about space exploration"}]
# No model specified - router will analyze and choose claude-3-5-sonnet-20241022
# No model specified - router will analyze and choose claude-sonnet-4-5
)
@ -237,17 +237,17 @@ You can combine static model selection with dynamic routing preferences for maxi
:caption: Hybrid Routing Configuration
llm_providers:
- model: openai/gpt-4o-mini
- model: openai/gpt-5.2
access_key: $OPENAI_API_KEY
default: true
- model: openai/gpt-4o
- model: openai/gpt-5
access_key: $OPENAI_API_KEY
routing_preferences:
- name: complex_reasoning
description: deep analysis and complex problem solving
- model: anthropic/claude-3-5-sonnet-20241022
- model: anthropic/claude-sonnet-4-5
access_key: $ANTHROPIC_API_KEY
routing_preferences:
- name: creative_tasks
@ -256,14 +256,14 @@ You can combine static model selection with dynamic routing preferences for maxi
model_aliases:
# Model aliases - friendly names that map to actual provider names
fast-model:
target: gpt-4o-mini
target: gpt-5.2
reasoning-model:
target: gpt-4o
target: gpt-5
# Aliases that can also participate in dynamic routing
creative-model:
target: claude-3-5-sonnet-20241022
target: claude-sonnet-4-5
This configuration allows clients to:

14
docs/source/guides/observability/access_logging.rst
View file

@ -19,24 +19,24 @@ Key Features
How It Works
^^^^^^^^^^^^
Plano exposes access logs for every call it manages on your behalf. By default these access logs can be found under ``~/archgw_logs``. For example:
Plano exposes access logs for every call it manages on your behalf. By default these access logs can be found under ``~/plano_logs``. For example:
.. code-block:: console
$ tail -F ~/archgw_logs/access_*.log
$ tail -F ~/plano_logs/access_*.log
==> /Users/adilhafeez/archgw_logs/access_llm.log <==
==> /Users/username/plano_logs/access_llm.log <==
[2024-10-10T03:55:49.537Z] "POST /v1/chat/completions HTTP/1.1" 0 DC 0 0 770 - "-" "OpenAI/Python 1.51.0" "469793af-b25f-9b57-b265-f376e8d8c586" "api.openai.com" "162.159.140.245:443"
==> /Users/adilhafeez/archgw_logs/access_internal.log <==
==> /Users/username/plano_logs/access_internal.log <==
[2024-10-10T03:56:03.906Z] "POST /embeddings HTTP/1.1" 200 - 52 21797 54 53 "-" "-" "604197fe-2a5b-95a2-9367-1d6b30cfc845" "model_server" "192.168.65.254:51000"
[2024-10-10T03:56:03.961Z] "POST /zeroshot HTTP/1.1" 200 - 106 218 87 87 "-" "-" "604197fe-2a5b-95a2-9367-1d6b30cfc845" "model_server" "192.168.65.254:51000"
[2024-10-10T03:56:04.050Z] "POST /v1/chat/completions HTTP/1.1" 200 - 1301 614 441 441 "-" "-" "604197fe-2a5b-95a2-9367-1d6b30cfc845" "model_server" "192.168.65.254:51000"
[2024-10-10T03:56:04.492Z] "POST /hallucination HTTP/1.1" 200 - 556 127 104 104 "-" "-" "604197fe-2a5b-95a2-9367-1d6b30cfc845" "model_server" "192.168.65.254:51000"
[2024-10-10T03:56:04.598Z] "POST /insurance_claim_details HTTP/1.1" 200 - 447 125 17 17 "-" "-" "604197fe-2a5b-95a2-9367-1d6b30cfc845" "api_server" "192.168.65.254:18083"
==> /Users/adilhafeez/archgw_logs/access_ingress.log <==
[2024-10-10T03:56:03.905Z] "POST /v1/chat/completions HTTP/1.1" 200 - 463 1022 1695 984 "-" "OpenAI/Python 1.51.0" "604197fe-2a5b-95a2-9367-1d6b30cfc845" "arch_llm_listener" "0.0.0.0:12000"
==> /Users/username/plano_logs/access_ingress.log <==
[2024-10-10T03:56:03.905Z] "POST /v1/chat/completions HTTP/1.1" 200 - 463 1022 1695 984 "-" "OpenAI/Python 1.51.0" "604197fe-2a5b-95a2-9367-1d6b30cfc845" "plano_llm_listener" "0.0.0.0:12000"
Log Format
@ -58,6 +58,6 @@ For example for following request:
.. code-block:: console
[2024-10-10T03:56:03.905Z] "POST /v1/chat/completions HTTP/1.1" 200 - 463 1022 1695 984 "-" "OpenAI/Python 1.51.0" "604197fe-2a5b-95a2-9367-1d6b30cfc845" "arch_llm_listener" "0.0.0.0:12000"
[2024-10-10T03:56:03.905Z] "POST /v1/chat/completions HTTP/1.1" 200 - 463 1022 1695 984 "-" "OpenAI/Python 1.51.0" "604197fe-2a5b-95a2-9367-1d6b30cfc845" "plano_llm_listener" "0.0.0.0:12000"
Total duration was 1695ms, and the upstream service took 984ms to process the request. Bytes received and sent were 463 and 1022 respectively.