mirror of
https://github.com/katanemo/plano.git
synced 2026-06-20 15:28:07 +02:00
several improvements to docs. TODOS: Tracing and Filters
This commit is contained in:
Salman Paracha
parent
1d6a1613a2
commit
e0404d305c
9 changed files with 297 additions and 301 deletions
100
README.md
100
README.md
|
|
@ -3,8 +3,8 @@
|
|||
</div>
|
||||
<div align="center">
|
||||
|
||||
_Plano is a models-native proxy and data plane for agents._<br><br>
|
||||
Plano pulls out the rote plumbing work and decouples you from brittle framework abstractions, centralizing what shouldn’t be bespoke in every codebase - like agent routing and orchestration, rich agentic signals and traces for continuous improvement, guardrail filters for moderation, and smart LLM routing APIs for UX and DX agility. Use any language or AI framework, and deliver agents faster to production.
|
||||
_Plano is a models-native proxy server and data plane for agents._<br><br>
|
||||
Plano pulls out the rote plumbing work and decouples you from brittle framework abstractions, centralizing what shouldn’t be bespoke in every codebase - like agent routing and orchestration, rich agentic signals and traces for continuous improvement, guardrail filters for safety and moderation, and smart LLM routing APIs for UX and DX agility. Use any language or AI framework, and deliver agents faster to production.
|
||||
|
||||
|
||||
[Quickstart](#Quickstart) •
|
||||
|
|
@ -26,7 +26,7 @@ Building agentic demos is easy. Shipping agentic applications safely, reliably,
|
|||
|
||||
Plano solves this by moving core delivery concerns into a unified, out-of-process dataplane.
|
||||
|
||||
- **🚦 Orchestration:** Low-latency orchestration between agents, and add new agents without changing app code
|
||||
- **🚦 Orchestration:** Low-latency orchestration between agents; add new agents without changing application code
|
||||
- **🔗 Model Agility:** Route [by model name, alias (semantic names) or automatically via preferences](#use-plano-as-a-llm-router)
|
||||
- **🕵 Agentic Signals™:** Zero-code capture of [behavior signals](#observability) plus OTEL traces/metrics across every agent.
|
||||
- **🛡️ Moderation & Memory Hooks:** Build jailbreak protection, add moderation policies and memory consistently via [Filter Chains](https://docs.planoai.dev/concepts/filter_chain.html).
|
||||
|
|
@ -75,8 +75,8 @@ $ pip install plano==0.4.0
|
|||
### Use Plano as a LLM Router
|
||||
Plano supports multiple powerful routing strategies for LLMs. [Model-based routing](https://docs.arch.com/guides/llm_router.html#model-based-routing) gives you direct control over specific models and supports 11+ LLM providers including OpenAI, Anthropic, DeepSeek, Mistral, Groq, and more. [Alias-based routing](https://docs.arch.com/guides/llm_router.html#alias-based-routing) lets you create semantic model names that decouple your application code from specific providers, making it easy to experiment with different models or handle provider changes without refactoring. For full configuration examples and code walkthroughs, see our [routing guides](https://docs.arch.com/guides/llm_router.html).
|
||||
|
||||
#### Preference-aligned Routing
|
||||
Preference-aligned routing provides intelligent, dynamic model selection based on natural language descriptions of tasks and preferences. Instead of hardcoded routing logic, you describe what each model is good at using plain English.
|
||||
#### Policy-based Routing
|
||||
Policy-based routing provides deterministic constructs to achieve automatic routing. intelligent, dynamic model selection based on natural language descriptions of tasks and preferences. Instead of hardcoded routing logic, you describe what each model is good at using plain English.
|
||||
|
||||
```yaml
|
||||
version: v0.1.0
|
||||
|
|
@ -106,8 +106,6 @@ llm_providers:
|
|||
description: analyzing existing code for bugs, improvements, and optimization
|
||||
```
|
||||
|
||||
|
||||
|
||||
Plano uses a lightweight 1.5B autoregressive model to intelligently map user prompts to these preferences, automatically selecting the best model for each request. This approach adapts to intent drift, supports multi-turn conversations, and avoids brittle embedding-based classifiers or manual if/else chains. No retraining required when adding models or updating policies — routing is governed entirely by human-readable rules.
|
||||
|
||||
**Learn More**: Check our [documentation](https://docs.plano.com/concepts/llm_providers/llm_providers.html) for comprehensive provider setup guides and routing strategies. You can learn more about the design, benchmarks, and methodology behind preference-based routing in our paper:
|
||||
|
|
@ -120,11 +118,78 @@ Plano uses a lightweight 1.5B autoregressive model to intelligently map user pro
|
|||
|
||||
### Build Agentic Apps with Plano
|
||||
|
||||
In following quickstart we will show you how easy it is to build AI agent with Plano gateway. We will build a currency exchange agent using following simple steps. For this demo we will use `https://api.frankfurter.dev/` to fetch latest price for currencies and assume USD as base currency.
|
||||
Plano helps you build agentic applications in two complementary ways:
|
||||
|
||||
#### Step 1. Create plano config file
|
||||
- **Orchestrate agents**: Let Plano decide which agent or LLM should handle each request and in what sequence.
|
||||
- **Call deterministic backends**: Use prompt targets to turn natural-language prompts into structured, validated API calls.
|
||||
|
||||
Create `plano_config.yaml` file with following content,
|
||||
You focus on product logic (agents and APIs) while Plano handles routing, parameter extraction, and wiring. The full examples used here are available in the [`plano-quickstart` repository](https://github.com/plano-ai/plano-quickstart).
|
||||
|
||||
#### Build agents with Plano orchestration
|
||||
|
||||
Agents are where your business logic lives (the "inner loop"). Plano takes care of the "outer loop"—routing, sequencing, and managing calls across agents and LLMs. In this quick example, we show a simplified **Travel Assistant** that routes between a `flight_agent` and a `hotel_agent`.
|
||||
|
||||
##### Step 1. Minimal orchestration config
|
||||
|
||||
Create a `plano_config.yaml` that wires Plano-Orchestrator to your agents:
|
||||
|
||||
```yaml
|
||||
version: v0.1.0
|
||||
|
||||
agents:
|
||||
- id: flight_agent
|
||||
url: http://host.docker.internal:10520 # your flights service
|
||||
- id: hotel_agent
|
||||
url: http://host.docker.internal:10530 # your hotels service
|
||||
|
||||
model_providers:
|
||||
- model: openai/gpt-4o
|
||||
access_key: $OPENAI_API_KEY
|
||||
|
||||
listeners:
|
||||
- type: agent
|
||||
name: travel_assistant
|
||||
port: 8001
|
||||
router: plano_orchestrator_v1
|
||||
agents:
|
||||
- id: flight_agent
|
||||
description: Search for flights and provide flight status.
|
||||
- id: hotel_agent
|
||||
description: Find hotels and check availability.
|
||||
|
||||
tracing:
|
||||
random_sampling: 100
|
||||
```
|
||||
|
||||
##### Step 2. Start your agents and Plano
|
||||
|
||||
Run your `flight_agent` and `hotel_agent` services (see the [Orchestration guide](https://docs.planoai.dev/guides/orchestration.html) for a full Travel Booking example), then start Plano with the config above:
|
||||
|
||||
```console
|
||||
$ plano up plano_config.yaml
|
||||
```
|
||||
|
||||
Plano will start the orchestrator and expose an agent listener on port `8001`.
|
||||
|
||||
##### Step 3. Send a prompt and let Plano route
|
||||
|
||||
Send a request to Plano using the OpenAI-compatible chat completions API—the orchestrator will analyze the prompt and route it to the right agent based on intent:
|
||||
|
||||
```bash
|
||||
$ curl --header 'Content-Type: application/json' \
|
||||
--data '{"messages": [{"role": "user","content": "Find me flights from SFO to JFK tomorrow"}], "model": "openai/gpt-4o"}' \
|
||||
http://localhost:8001/v1/chat/completions
|
||||
```
|
||||
|
||||
You can then ask a follow-up like "Also book me a hotel near JFK" and Plano-Orchestrator will route to `hotel_agent`—your agents stay focused on business logic while Plano handles routing.
|
||||
|
||||
#### Deterministic API calls with prompt targets
|
||||
|
||||
Next, we'll show Plano's deterministic API calling using a single prompt target. We'll build a currency exchange backend powered by `https://api.frankfurter.dev/`, assuming USD as the base currency.
|
||||
|
||||
##### Step 1. Create plano config file
|
||||
|
||||
Create `plano_config.yaml` with the following content:
|
||||
|
||||
```yaml
|
||||
version: v0.1.0
|
||||
|
|
@ -170,10 +235,9 @@ endpoints:
|
|||
protocol: https
|
||||
```
|
||||
|
||||
#### Step 2. Start plano gateway with currency conversion config
|
||||
##### Step 2. Start Plano with currency conversion config
|
||||
|
||||
```sh
|
||||
|
||||
$ plano up plano_config.yaml
|
||||
2024-12-05 16:56:27,979 - cli.main - INFO - Starting plano cli version: 0.4.0
|
||||
2024-12-05 16:56:28,485 - cli.utils - INFO - Schema validation successful!
|
||||
|
|
@ -181,13 +245,13 @@ $ plano up plano_config.yaml
|
|||
2024-12-05 16:56:51,647 - cli.core - INFO - Container is healthy!
|
||||
```
|
||||
|
||||
Once the gateway is up you can start interacting with at port 10000 using openai chat completion API.
|
||||
Once the gateway is up you can start interacting with it at port `10000` using the OpenAI chat completion API.
|
||||
|
||||
Some of the sample queries you can ask could be `what is currency rate for gbp?` or `show me list of currencies for conversion`.
|
||||
Some sample queries you can ask include: `what is currency rate for gbp?` or `show me list of currencies for conversion`.
|
||||
|
||||
#### Step 3. Interacting with gateway using curl command
|
||||
##### Step 3. Interact with the gateway using curl
|
||||
|
||||
Here is a sample curl command you can use to interact,
|
||||
Here is a sample curl command you can use to interact:
|
||||
|
||||
```bash
|
||||
$ curl --header 'Content-Type: application/json' \
|
||||
|
|
@ -195,10 +259,9 @@ $ curl --header 'Content-Type: application/json' \
|
|||
http://localhost:10000/v1/chat/completions | jq ".choices[0].message.content"
|
||||
|
||||
"As of the date provided in your context, December 5, 2024, the exchange rate for GBP (British Pound) from USD (United States Dollar) is 0.78558. This means that 1 USD is equivalent to 0.78558 GBP."
|
||||
|
||||
```
|
||||
|
||||
And to get list of supported currencies,
|
||||
And to get the list of supported currencies:
|
||||
|
||||
```bash
|
||||
$ curl --header 'Content-Type: application/json' \
|
||||
|
|
@ -206,7 +269,6 @@ $ curl --header 'Content-Type: application/json' \
|
|||
http://localhost:10000/v1/chat/completions | jq ".choices[0].message.content"
|
||||
|
||||
"Here is a list of the currencies that are supported for conversion from USD, along with their symbols:\n\n1. AUD - Australian Dollar\n2. BGN - Bulgarian Lev\n3. BRL - Brazilian Real\n4. CAD - Canadian Dollar\n5. CHF - Swiss Franc\n6. CNY - Chinese Renminbi Yuan\n7. CZK - Czech Koruna\n8. DKK - Danish Krone\n9. EUR - Euro\n10. GBP - British Pound\n11. HKD - Hong Kong Dollar\n12. HUF - Hungarian Forint\n13. IDR - Indonesian Rupiah\n14. ILS - Israeli New Sheqel\n15. INR - Indian Rupee\n16. ISK - Icelandic Króna\n17. JPY - Japanese Yen\n18. KRW - South Korean Won\n19. MXN - Mexican Peso\n20. MYR - Malaysian Ringgit\n21. NOK - Norwegian Krone\n22. NZD - New Zealand Dollar\n23. PHP - Philippine Peso\n24. PLN - Polish Złoty\n25. RON - Romanian Leu\n26. SEK - Swedish Krona\n27. SGD - Singapore Dollar\n28. THB - Thai Baht\n29. TRY - Turkish Lira\n30. USD - United States Dollar\n31. ZAR - South African Rand\n\nIf you want to convert USD to any of these currencies, you can select the one you are interested in."
|
||||
|
||||
```
|
||||
|
||||
## [Observability](https://docs.plano.com/guides/observability/observability.html)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue