apunkt/plano
1
0
Fork 0
mirror of https://github.com/katanemo/plano.git synced 2026-06-20 15:28:07 +02:00
Code Issues Projects Releases Packages Wiki Activity
plano/demos/llm_routing/preference_based_routing
History
Adil Hafeez 056ed67a04
Update documentation and test references for new demo paths 
- Update README.md demo path references to agent_orchestration/
- Update docs/source paths for multi_turn_rag and travel_agents
- Update e2e test scripts for weather_forecast and model_alias_routing
- Update docker-compose.dev.yaml and e2e docker-compose.yaml paths
- Update demo READMEs with correct cross-references and remove
  references to deleted observability stacks
- Update test runner script for preference_based_routing path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 01:44:16 -08:00
..
hurl_tests Clean up dead demos and restructure directory layout 2026-02-15 01:43:31 -08:00
config.yaml Standardize demo configs and CLI templates to v0.3.0 2026-02-15 01:43:59 -08:00
docker-compose.yaml Standardize demo configs and CLI templates to v0.3.0 2026-02-15 01:43:59 -08:00
plano_config_local.yaml Standardize demo configs and CLI templates to v0.3.0 2026-02-15 01:43:59 -08:00
README.md Update documentation and test references for new demo paths 2026-02-15 01:44:16 -08:00
test_router_endpoint.rest Clean up dead demos and restructure directory layout 2026-02-15 01:43:31 -08:00

README.md

Usage based LLM Routing

This demo shows how you can use user preferences to route user prompts to appropriate llm. See config.yaml for details on how you can define user preferences.

How to start the demo

Make sure your machine is up to date with latest version of plano. And you have activated the virtual environment.

  1. start anythingllm
(venv) $ cd demos/llm_routing/preference_based_routing
(venv) $ docker compose up -d
  1. start plano in the foreground
(venv) $ planoai up --service plano --foreground
# Or if installed with uv: uvx planoai up --service plano --foreground
2025-05-30 18:00:09,953 - planoai.main - INFO - Starting plano cli version: 0.4.6
2025-05-30 18:00:09,953 - planoai.main - INFO - Validating /Users/adilhafeez/src/intelligent-prompt-gateway/demos/llm_routing/preference_based_routing/config.yaml
2025-05-30 18:00:10,422 - cli.core - INFO - Starting plano gateway, image name: plano, tag: katanemo/plano:0.4.6
2025-05-30 18:00:10,662 - cli.core - INFO - plano status: running, health status: starting
2025-05-30 18:00:11,712 - cli.core - INFO - plano status: running, health status: starting
2025-05-30 18:00:12,761 - cli.core - INFO - plano is running and is healthy!
...
  1. open AnythingLLM http://localhost:3001/

Testing out preference based routing

We have defined two routes 1. code generation and 2. code understanding

For code generation query LLM that is better suited for code generation wil handle the request,

If you look at the logs you'd see that code generation llm was selected,

...
2025-05-31T01:02:19.382716Z  INFO brightstaff::router::llm_router: router response: {'route': 'code_generation'}, response time: 203ms
...
image

Now if you ask for query related to code understanding you'd see llm that is better suited to handle code understanding in handled,

...
2025-05-31T01:06:33.555680Z  INFO brightstaff::router::llm_router: router response: {'route': 'code_understanding'}, response time: 327ms
...
image