apunkt/plano
1
0
Fork 0
mirror of https://github.com/katanemo/plano.git synced 2026-07-23 16:51:04 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat(routing): decouple budget pricing from prompt_caching (#996)

Browse source 
* feat(routing): budget prices cache-aware regardless of prompt_caching flag

* docs: prompt_caching is optional for the routing budget
This commit is contained in:
Adil Hafeez 2026-07-22 13:55:48 -07:00 • committed by GitHub
parent 66547bcef3
commit ad92f7fbe7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 69 additions and 124 deletions
Download patch file Download diff file Expand all files Collapse all files

7
crates/common/src/configuration.rs
View file

@ -33,9 +33,10 @@ pub struct Routing {
pub session_ttl_seconds: Option<u64>,
pub session_max_entries: Option<usize>,
pub session_cache: Option<SessionCacheConfig>,
/// Cost gate on model switching within a session. Independent of prompt caching:
/// this is a routing decision that applies whenever it is configured, whether or
/// not `prompt_caching` is enabled. Presence of this block turns it on.
/// Cost gate on model switching within a session. Self-sufficient and independent
/// of prompt caching: presence of this block turns it on, implicit sessions are
/// derived on its own, and warm anchors are always priced at cached rates —
/// `prompt_caching` only controls marker injection and affinity-without-budget.
pub routing_budget: Option<RoutingBudget>,
}