* feat(routing): cache-aware routing and zero-config prompt caching
Reconcile prompt caching with intelligent routing: zero-config implicit
session affinity, cache_control preservation/injection across transforms,
cache-adjusted routing economics, and a response-driven cache-hit feedback loop.
* refactor(caching): model-aware cache markers, drop cache-aware routing
* feat(routing): session stickiness cache-regret cost gate
* refactor(routing): expose per-model input and cached rates
* fix(docker): patch libssh2 CVEs by upgrading after package install
* feat: record counterfactual route on vetoed session switches
Add opt-in session_stickiness.record_counterfactual that emits the
plano.switch.counterfactual_route OTEL attribute when the cache-regret
gate retains the previous model, capturing the route it would have taken
had the switch been allowed. Telemetry only; the candidate is never
dispatched. Includes schema, demo config, and a getting-started guide.
* refactor: group prompt-caching and session-stickiness logic into cohesive modules
Extract the two concerns interleaved in the LLM handler into dedicated
sibling modules for readability: prompt_caching.rs (cache-marker injection)
and session_stickiness.rs (session key/prefix-hash resolution, pin lookup,
cache-regret gate, and pin planning). handlers/llm/mod.rs is now a thin
orchestrator that calls them at clear phase boundaries. Behavior-preserving.
* refactor(routing): unify session routing into a cumulative switch budget
Replace observed_cache_hit + per-request threshold with time/provider-TTL
warmth and a per-session switch budget in a single session_router::route()
used by both the full-proxy and /routing decision paths.
* refactor(routing): move switch budget to routing.routing_budget and rename telemetry
* refactor(routing): drop negative-cost budget refund from routing_budget
* refactor(routing): express routing_budget as a percentage overhead cap
Replace the absolute seed_usd pool with max_overhead_pct (whole-number
percent) measured against the session's running never-switch baseline:
allow a paid switch only while cumulative switch spend stays within
max_overhead_pct% of what staying on the anchor would have cost. Track
baseline_usd and switch_spend_usd on the binding (monotonic, no refunds).
* refactor(observability): align routing-budget telemetry with the overhead cap
Rename the switch-decision reasons (within_budget/over_budget -> within_cap/
over_cap) and the session span attributes to match the percentage overhead-cap
model: budget_remaining_in_usd -> overhead_pct + switch_spend_in_usd +
baseline_in_usd, and switch threshold_in_usd -> overhead_ceiling_in_usd.
* feat(observability): emit per-request and cumulative session cost
Price each turn from the catalog rates and surface it as llm.usage.{input,
output,total}_cost_usd on the (llm) span, then accumulate into a conversation-
level session_cost_usd on the binding and emit plano.session.total_cost_in_usd
on the routing span. Cache-creation tokens are priced at the plain input rate,
and the OpenAI vs Anthropic prompt-token convention (cached folded in vs
reported separately) is captured at parse time so the cost math is correct for
both. Rates are resolved request-side since the response path is synchronous.
* refactor(routing): price the never-switch baseline against the session default model
Distinguish the session's default_model (the model it started on, i.e. what it
would have cost by never switching) from anchor_model (the model that handled
the latest request, which the session is warm on). The never-switch baseline now
grows at the default model's cached rate rather than the drifting anchor's, so
the overhead-cap denominator stays true to "% above never-switching"; switch cost
is still measured against the current anchor. Also rename estimate_context_tokens
-> actual_context_tokens (and est_context_tokens -> context_tokens) since the
router prefers the provider's real prompt-token count when the session is warm.
* feat(routing): track bounded route history and price returns to still-warm models
Record a bounded (LRU, capped) per-model visit history on the session binding
and use it to sharpen the switch-cost estimate: a return to a model still within
its cache window re-reads only the tokens appended since its last visit (at its
cached rate) instead of the whole context at the uncached rate, so an A->B->A
switch is no longer over-charged as a full re-ingest. History is carried through
the response-side refresh (refining the anchor's entry from real usage) and
cleared on prefix drift. Adds the plano.switch.candidate_warm_tokens span
attribute.
* refactor(routing): tidy test names and rename pin events to binding events
Shorten run-on test names and replace stale "pin" vocabulary in the session
binding metric (brightstaff_session_pin_events_total -> _binding_events_total),
dropping lifecycle event labels that are no longer emitted.
* feat(routing): price switch cost against uncached rates when caching is off
The overhead-cap gate previously always priced the anchor at its cached rate,
assuming a warm provider cache. When prompt caching is disabled there is no
warm cache to lose, so both the switch cost and the never-switch baseline are
now priced at the plain uncached input rate:
switch_cost = context_tokens x (candidate_uncached - anchor_uncached) / 1M.
* fix(brightstaff): bound pricing-feed fetch with timeouts and warn when session state runs without a tenant header
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(routing): validate the session overhead cap end-to-end across a multi-turn session
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(demos): fix stale pinning semantics and add routing_budget demo script
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Adil Hafeez <adil.hafeez@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(routing): auto-migrate v0.3.0 inline routing_preferences to v0.4.0 top-level
Lift inline routing_preferences under each model_provider into the
top-level routing_preferences list with merged models[] and bump
version to v0.4.0, with a deprecation warning. Existing v0.3.0
demo configs (Claude Code, Codex, preference_based_routing, etc.)
keep working unchanged. Schema flags the inline shape as deprecated
but still accepts it. Docs and skills updated to canonical top-level
multi-model form.
* test(common): bump reference config assertion to v0.4.0
The rendered reference config was bumped to v0.4.0 when its inline
routing_preferences were lifted to the top level; align the
configuration deserialization test with that change.
* fix(config_generator): bump version to v0.4.0 up front in migration
Move the v0.3.0 -> v0.4.0 version bump to the top of
migrate_inline_routing_preferences so it runs unconditionally,
including for configs that already declare top-level
routing_preferences at v0.3.0. Previously the bump only fired
when inline migration produced entries, leaving top-level v0.3.0
configs rejected by brightstaff's v0.4.0 gate. Tests updated to
cover the new behavior and to confirm we never downgrade newer
versions.
* fix(config_generator): gate routing_preferences migration on version < v0.4.0
Short-circuit the migration when the config already declares v0.4.0
or newer. Anything at v0.4.0+ is assumed to be on the canonical
top-level shape and is passed through untouched, including stray
inline preferences (which are the author's bug to fix). Only v0.3.0
and older configs are rewritten and bumped.
* support configurable orchestrator model via orchestration config section
* add self-hosting docs and demo for Plano-Orchestrator
* list all Plano-Orchestrator model variants in docs
* use overrides for custom routing and orchestration model
* update docs
* update orchestrator model name
* rename arch provider to plano, use llm_routing_model and agent_orchestration_model
* regenerate rendered config reference
* Add Codex CLI support; xAI response improvements
* Add native Plano running check and update CLI agent error handling
* adding PR suggestions for transformations and code quality
* message extraction logic in ResponsesAPIRequest
* xAI support for Responses API by routing to native endpoint + refactor code
* Add OpenClaw + Plano intelligent routing demo
Demonstrates preference-based routing for personal AI assistants:
Kimi K2.5 handles conversation and agentic tasks, Claude handles
code generation, testing, and complex reasoning — with zero
application code changes and ~48% cost savings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove redundant provider_interface from Kimi K2.5 config
The openai/ prefix in the model name already sets the provider
interface. Setting provider_interface explicitly conflicts with it
and fails config validation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Simplify config to v0.3.0 format, remove explicit Arch-Router entry
Arch-Router is implicit when routing_preferences are defined.
Aligns with the preference_based_routing demo pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Clean up Ollama/Arch-Router references, make Jaeger optional
Router is handled internally by Plano — no need for Ollama or
explicit Arch-Router setup. Jaeger is kept as an optional step
in the README for developers who want tracing visibility.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove run_demo.sh, use planoai CLI directly
The planoai CLI already handles startup. README now uses
planoai up/down directly instead of a wrapper script.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove docker-compose.yaml, use inline docker run for Jaeger
No need for a compose file when Jaeger is the only optional
service. A single docker run command in the README is simpler.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Clarify testing: OpenClaw channels vs direct Plano requests
Primary testing is through messaging channels (Telegram, Slack,
etc.) with log monitoring. The test_routing.sh script is now
documented as an optional direct verification tool.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add OpenClaw onboarding instructions to README
Includes install, onboarding wizard, channel setup, doctor
check, and how to point the gateway at Plano.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Use OpenClaw onboarding wizard for Plano provider setup
Replace manual JSON config with instructions to use the
openclaw onboard wizard to set up a custom OpenAI-compatible
provider pointing at Plano.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fixed readme and removed unnecessary testing.sh file
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-342.local>
Demonstrates preference-based routing for personal AI assistants:
Kimi K2.5 handles conversation and agentic tasks, Claude handles
code generation, testing, and complex reasoning — with zero
application code changes and ~48% cost savings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update all GitHub Actions workflows and Dockerfiles to use Python 3.14
as the default version. Remove the upper bound on requires-python in
model_choice_with_test_harness to allow 3.14+. The CLI's
requires-python stays at >=3.10 for broad compatibility.
