mirror of
https://github.com/katanemo/plano.git
synced 2026-07-23 16:51:04 +02:00
844f08bda7
* feat(routing): cache-aware routing and zero-config prompt caching
Reconcile prompt caching with intelligent routing: zero-config implicit
session affinity, cache_control preservation/injection across transforms,
cache-adjusted routing economics, and a response-driven cache-hit feedback loop.
* refactor(caching): model-aware cache markers, drop cache-aware routing
* feat(routing): session stickiness cache-regret cost gate
* refactor(routing): expose per-model input and cached rates
* fix(docker): patch libssh2 CVEs by upgrading after package install
* feat: record counterfactual route on vetoed session switches
Add opt-in session_stickiness.record_counterfactual that emits the
plano.switch.counterfactual_route OTEL attribute when the cache-regret
gate retains the previous model, capturing the route it would have taken
had the switch been allowed. Telemetry only; the candidate is never
dispatched. Includes schema, demo config, and a getting-started guide.
* refactor: group prompt-caching and session-stickiness logic into cohesive modules
Extract the two concerns interleaved in the LLM handler into dedicated
sibling modules for readability: prompt_caching.rs (cache-marker injection)
and session_stickiness.rs (session key/prefix-hash resolution, pin lookup,
cache-regret gate, and pin planning). handlers/llm/mod.rs is now a thin
orchestrator that calls them at clear phase boundaries. Behavior-preserving.
* refactor(routing): unify session routing into a cumulative switch budget
Replace observed_cache_hit + per-request threshold with time/provider-TTL
warmth and a per-session switch budget in a single session_router::route()
used by both the full-proxy and /routing decision paths.
* refactor(routing): move switch budget to routing.routing_budget and rename telemetry
* refactor(routing): drop negative-cost budget refund from routing_budget
* refactor(routing): express routing_budget as a percentage overhead cap
Replace the absolute seed_usd pool with max_overhead_pct (whole-number
percent) measured against the session's running never-switch baseline:
allow a paid switch only while cumulative switch spend stays within
max_overhead_pct% of what staying on the anchor would have cost. Track
baseline_usd and switch_spend_usd on the binding (monotonic, no refunds).
* refactor(observability): align routing-budget telemetry with the overhead cap
Rename the switch-decision reasons (within_budget/over_budget -> within_cap/
over_cap) and the session span attributes to match the percentage overhead-cap
model: budget_remaining_in_usd -> overhead_pct + switch_spend_in_usd +
baseline_in_usd, and switch threshold_in_usd -> overhead_ceiling_in_usd.
* feat(observability): emit per-request and cumulative session cost
Price each turn from the catalog rates and surface it as llm.usage.{input,
output,total}_cost_usd on the (llm) span, then accumulate into a conversation-
level session_cost_usd on the binding and emit plano.session.total_cost_in_usd
on the routing span. Cache-creation tokens are priced at the plain input rate,
and the OpenAI vs Anthropic prompt-token convention (cached folded in vs
reported separately) is captured at parse time so the cost math is correct for
both. Rates are resolved request-side since the response path is synchronous.
* refactor(routing): price the never-switch baseline against the session default model
Distinguish the session's default_model (the model it started on, i.e. what it
would have cost by never switching) from anchor_model (the model that handled
the latest request, which the session is warm on). The never-switch baseline now
grows at the default model's cached rate rather than the drifting anchor's, so
the overhead-cap denominator stays true to "% above never-switching"; switch cost
is still measured against the current anchor. Also rename estimate_context_tokens
-> actual_context_tokens (and est_context_tokens -> context_tokens) since the
router prefers the provider's real prompt-token count when the session is warm.
* feat(routing): track bounded route history and price returns to still-warm models
Record a bounded (LRU, capped) per-model visit history on the session binding
and use it to sharpen the switch-cost estimate: a return to a model still within
its cache window re-reads only the tokens appended since its last visit (at its
cached rate) instead of the whole context at the uncached rate, so an A->B->A
switch is no longer over-charged as a full re-ingest. History is carried through
the response-side refresh (refining the anchor's entry from real usage) and
cleared on prefix drift. Adds the plano.switch.candidate_warm_tokens span
attribute.
* refactor(routing): tidy test names and rename pin events to binding events
Shorten run-on test names and replace stale "pin" vocabulary in the session
binding metric (brightstaff_session_pin_events_total -> _binding_events_total),
dropping lifecycle event labels that are no longer emitted.
* feat(routing): price switch cost against uncached rates when caching is off
The overhead-cap gate previously always priced the anchor at its cached rate,
assuming a warm provider cache. When prompt caching is disabled there is no
warm cache to lose, so both the switch cost and the never-switch baseline are
now priced at the plain uncached input rate:
switch_cost = context_tokens x (candidate_uncached - anchor_uncached) / 1M.
* fix(brightstaff): bound pricing-feed fetch with timeouts and warn when session state runs without a tenant header
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(routing): validate the session overhead cap end-to-end across a multi-turn session
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(demos): fix stale pinning semantics and add routing_budget demo script
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Adil Hafeez <adil.hafeez@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
730 lines
22 KiB
YAML
730 lines
22 KiB
YAML
$schema: 'http://json-schema.org/draft-07/schema#'
|
|
type: object
|
|
properties:
|
|
version:
|
|
type: string
|
|
enum:
|
|
- v0.1
|
|
- v0.1.0
|
|
- 0.1-beta
|
|
- 0.2.0
|
|
- v0.3.0
|
|
- v0.4.0
|
|
|
|
agents:
|
|
type: array
|
|
items:
|
|
type: object
|
|
properties:
|
|
id:
|
|
type: string
|
|
url:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- id
|
|
- url
|
|
filters:
|
|
type: array
|
|
items:
|
|
type: object
|
|
properties:
|
|
id:
|
|
type: string
|
|
url:
|
|
type: string
|
|
type:
|
|
type: string
|
|
enum:
|
|
- mcp
|
|
- http
|
|
transport:
|
|
type: string
|
|
enum:
|
|
- streamable-http
|
|
tool:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- id
|
|
- url
|
|
listeners:
|
|
oneOf:
|
|
- type: array
|
|
additionalProperties: false
|
|
items:
|
|
type: object
|
|
properties:
|
|
name:
|
|
type: string
|
|
port:
|
|
type: integer
|
|
address:
|
|
type: string
|
|
timeout:
|
|
type: string
|
|
router:
|
|
type: string
|
|
enum:
|
|
- plano_orchestrator_v1
|
|
max_retries:
|
|
type: integer
|
|
type:
|
|
type: string
|
|
enum:
|
|
- model
|
|
- prompt
|
|
- agent
|
|
agents:
|
|
type: array
|
|
items:
|
|
type: object
|
|
properties:
|
|
id:
|
|
type: string
|
|
description:
|
|
type: string
|
|
default:
|
|
type: boolean
|
|
input_filters:
|
|
type: array
|
|
items:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- id
|
|
- description
|
|
input_filters:
|
|
type: array
|
|
items:
|
|
type: string
|
|
output_filters:
|
|
type: array
|
|
items:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- type
|
|
- name
|
|
- type: object # deprecated legacy format, use list format instead
|
|
additionalProperties: false
|
|
properties:
|
|
ingress_traffic:
|
|
type: object
|
|
properties:
|
|
address:
|
|
type: string
|
|
port:
|
|
type: integer
|
|
message_format:
|
|
type: string
|
|
enum:
|
|
- openai
|
|
timeout:
|
|
type: string
|
|
additionalProperties: false
|
|
egress_traffic:
|
|
type: object
|
|
properties:
|
|
address:
|
|
type: string
|
|
port:
|
|
type: integer
|
|
message_format:
|
|
type: string
|
|
enum:
|
|
- openai
|
|
timeout:
|
|
type: string
|
|
additionalProperties: false
|
|
endpoints:
|
|
type: object
|
|
patternProperties:
|
|
'^[a-zA-Z][a-zA-Z0-9_]*$':
|
|
type: object
|
|
properties:
|
|
endpoint:
|
|
type: string
|
|
pattern: '^.*$'
|
|
connect_timeout:
|
|
type: string
|
|
protocol:
|
|
type: string
|
|
enum:
|
|
- http
|
|
- https
|
|
http_host:
|
|
type: string
|
|
prefix_affinity:
|
|
type: boolean
|
|
description: >
|
|
For self-hosted multi-replica backends (e.g. vLLM): consistent-hash requests
|
|
to replicas by the x-plano-prefix-hash header so the same prompt prefix lands
|
|
on the same replica and reuses its KV cache. Uses ring-hash load balancing
|
|
across all resolved endpoint addresses. Default false.
|
|
additionalProperties: false
|
|
required:
|
|
- endpoint
|
|
|
|
model_providers:
|
|
type: array
|
|
items:
|
|
type: object
|
|
properties:
|
|
name:
|
|
type: string
|
|
access_key:
|
|
type: string
|
|
model:
|
|
type: string
|
|
default:
|
|
type: boolean
|
|
base_url:
|
|
type: string
|
|
passthrough_auth:
|
|
type: boolean
|
|
description: "When true, forwards the client's Authorization header to upstream instead of using the configured access_key. Useful for routing to services like LiteLLM that validate their own virtual keys."
|
|
http_host:
|
|
type: string
|
|
provider_interface:
|
|
type: string
|
|
enum:
|
|
- plano
|
|
- claude
|
|
- deepseek
|
|
- groq
|
|
- mistral
|
|
- openai
|
|
- xiaomi
|
|
- gemini
|
|
- chatgpt
|
|
- digitalocean
|
|
- vercel
|
|
- openrouter
|
|
- moonshotai
|
|
headers:
|
|
type: object
|
|
additionalProperties:
|
|
type: string
|
|
description: "Additional headers to send with upstream requests (e.g., ChatGPT-Account-Id, originator)."
|
|
routing_preferences:
|
|
type: array
|
|
description: "[DEPRECATED] Inline routing_preferences under a model_provider are auto-migrated to the top-level routing_preferences list by the config generator. New configs should declare routing_preferences at the top level with an explicit models: [...] list. See docs/routing-api.md."
|
|
items:
|
|
type: object
|
|
properties:
|
|
name:
|
|
type: string
|
|
description:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- name
|
|
- description
|
|
additionalProperties: false
|
|
required:
|
|
- model
|
|
|
|
llm_providers: # deprecated for legacy support, use model_providers instead
|
|
type: array
|
|
items:
|
|
type: object
|
|
properties:
|
|
name:
|
|
type: string
|
|
access_key:
|
|
type: string
|
|
model:
|
|
type: string
|
|
default:
|
|
type: boolean
|
|
base_url:
|
|
type: string
|
|
passthrough_auth:
|
|
type: boolean
|
|
description: "When true, forwards the client's Authorization header to upstream instead of using the configured access_key. Useful for routing to services like LiteLLM that validate their own virtual keys."
|
|
http_host:
|
|
type: string
|
|
provider_interface:
|
|
type: string
|
|
enum:
|
|
- plano
|
|
- claude
|
|
- deepseek
|
|
- groq
|
|
- mistral
|
|
- openai
|
|
- xiaomi
|
|
- gemini
|
|
- chatgpt
|
|
- digitalocean
|
|
- vercel
|
|
- openrouter
|
|
- moonshotai
|
|
headers:
|
|
type: object
|
|
additionalProperties:
|
|
type: string
|
|
description: "Additional headers to send with upstream requests (e.g., ChatGPT-Account-Id, originator)."
|
|
routing_preferences:
|
|
type: array
|
|
description: "[DEPRECATED] Inline routing_preferences under an llm_provider are auto-migrated to the top-level routing_preferences list by the config generator. New configs should declare routing_preferences at the top level with an explicit models: [...] list. See docs/routing-api.md."
|
|
items:
|
|
type: object
|
|
properties:
|
|
name:
|
|
type: string
|
|
description:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- name
|
|
- description
|
|
additionalProperties: false
|
|
required:
|
|
- model
|
|
|
|
model_aliases:
|
|
type: object
|
|
patternProperties:
|
|
'^.*$':
|
|
type: object
|
|
properties:
|
|
target:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- target
|
|
|
|
overrides:
|
|
type: object
|
|
properties:
|
|
prompt_target_intent_matching_threshold:
|
|
type: number
|
|
optimize_context_window:
|
|
type: boolean
|
|
use_agent_orchestrator:
|
|
type: boolean
|
|
disable_signals:
|
|
type: boolean
|
|
description: "Disable agentic signal analysis (frustration, repetition, escalation, etc.) on LLM responses to save CPU. Default false."
|
|
upstream_connect_timeout:
|
|
type: string
|
|
description: "Connect timeout for upstream provider clusters (e.g., '5s', '10s'). Default is '5s'."
|
|
upstream_tls_ca_path:
|
|
type: string
|
|
description: "Path to the trusted CA bundle for upstream TLS verification. Default is '/etc/ssl/certs/ca-certificates.crt'."
|
|
llm_routing_model:
|
|
type: string
|
|
description: "Model name for the LLM router (e.g., 'Plano-Orchestrator'). Must match a model in model_providers."
|
|
agent_orchestration_model:
|
|
type: string
|
|
description: "Model name for the agent orchestrator (e.g., 'Plano-Orchestrator'). Must match a model in model_providers."
|
|
orchestrator_model_context_length:
|
|
type: integer
|
|
description: "Maximum token length for the orchestrator/routing model context window. Default is 8192."
|
|
prompt_caching:
|
|
type: object
|
|
description: >
|
|
Automatic provider prompt caching, configured once for the whole Plano instance.
|
|
Disabled by default; set enabled: true to opt in. Prompt caching never changes
|
|
which model routing selects — it only keeps a conversation on the same
|
|
model/provider so the upstream prompt cache stays warm across turns, and
|
|
auto-injects provider cache-control markers where supported.
|
|
properties:
|
|
enabled:
|
|
type: boolean
|
|
description: "Master switch. Default false (opt-in). Applies across the entire instance."
|
|
session_affinity:
|
|
type: boolean
|
|
description: "Auto-derive a session key from the prompt prefix (system + tools + first user message) when X-Model-Affinity is absent, so follow-up turns reuse the same warm cache. Default true when enabled."
|
|
inject_cache_control:
|
|
type: boolean
|
|
description: "Auto-insert ephemeral cache_control breakpoints for providers that need explicit markers (Anthropic). Default true when enabled."
|
|
min_prefix_tokens:
|
|
type: integer
|
|
minimum: 1
|
|
description: "Skip breakpoint injection when the estimated stable prefix is below this many tokens. Default 1024."
|
|
session_ttl_seconds:
|
|
type: integer
|
|
minimum: 1
|
|
description: "Pin lifetime for implicit/explicit sessions; align with the provider cache window (e.g. 3600 for Anthropic 1h caching). Defaults to routing.session_ttl_seconds."
|
|
additionalProperties: false
|
|
system_prompt:
|
|
type: string
|
|
prompt_targets:
|
|
type: array
|
|
items:
|
|
type: object
|
|
properties:
|
|
name:
|
|
type: string
|
|
default:
|
|
type: boolean
|
|
description:
|
|
type: string
|
|
auto_llm_dispatch_on_response:
|
|
type: boolean
|
|
parameters:
|
|
type: array
|
|
items:
|
|
type: object
|
|
properties:
|
|
name:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
type: boolean
|
|
default:
|
|
anyOf:
|
|
- type: string
|
|
- type: integer
|
|
- type: boolean
|
|
description:
|
|
type: string
|
|
type:
|
|
type: string
|
|
enum:
|
|
type: array
|
|
items:
|
|
anyOf:
|
|
- type: string
|
|
- type: integer
|
|
- type: boolean
|
|
in_path:
|
|
type: boolean
|
|
format:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- name
|
|
- description
|
|
- type
|
|
endpoint:
|
|
type: object
|
|
properties:
|
|
name:
|
|
type: string
|
|
path:
|
|
type: string
|
|
http_method:
|
|
type: string
|
|
enum:
|
|
- GET
|
|
- POST
|
|
http_headers:
|
|
type: object
|
|
additionalProperties:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- name
|
|
- path
|
|
system_prompt:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- name
|
|
- description
|
|
ratelimits:
|
|
type: array
|
|
items:
|
|
type: object
|
|
properties:
|
|
model:
|
|
type: string
|
|
selector:
|
|
type: object
|
|
properties:
|
|
key:
|
|
type: string
|
|
value:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- key
|
|
- value
|
|
limit:
|
|
type: object
|
|
properties:
|
|
tokens:
|
|
type: integer
|
|
unit:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- tokens
|
|
- unit
|
|
additionalProperties: false
|
|
required:
|
|
- model
|
|
- selector
|
|
- limit
|
|
tracing:
|
|
type: object
|
|
properties:
|
|
random_sampling:
|
|
type: integer
|
|
trace_arch_internal:
|
|
type: boolean
|
|
opentracing_grpc_endpoint:
|
|
type: string
|
|
span_attributes:
|
|
type: object
|
|
properties:
|
|
header_prefixes:
|
|
type: array
|
|
items:
|
|
type: string
|
|
static:
|
|
type: object
|
|
additionalProperties:
|
|
type: string
|
|
additionalProperties: false
|
|
exporters:
|
|
type: array
|
|
items:
|
|
oneOf:
|
|
- type: object
|
|
properties:
|
|
type:
|
|
type: string
|
|
const: posthog
|
|
url:
|
|
type: string
|
|
api_key:
|
|
type: string
|
|
distinct_id_header:
|
|
type: string
|
|
capture_messages:
|
|
type: boolean
|
|
additionalProperties: false
|
|
required:
|
|
- type
|
|
- url
|
|
- api_key
|
|
additionalProperties: false
|
|
mode:
|
|
type: string
|
|
enum:
|
|
- llm
|
|
- prompt
|
|
routing:
|
|
type: object
|
|
properties:
|
|
llm_provider:
|
|
type: string
|
|
model:
|
|
type: string
|
|
session_ttl_seconds:
|
|
type: integer
|
|
minimum: 1
|
|
description: TTL in seconds for session-pinned routing cache entries. Default 600 (10 minutes).
|
|
session_max_entries:
|
|
type: integer
|
|
minimum: 1
|
|
maximum: 10000
|
|
description: Maximum number of session-pinned routing cache entries. Default 10000.
|
|
session_cache:
|
|
type: object
|
|
properties:
|
|
type:
|
|
type: string
|
|
enum:
|
|
- memory
|
|
- redis
|
|
default: memory
|
|
description: Session cache backend. "memory" (default) is in-process; "redis" is shared across replicas.
|
|
url:
|
|
type: string
|
|
description: Redis URL, e.g. redis://localhost:6379. Required when type is redis.
|
|
tenant_header:
|
|
type: string
|
|
description: >
|
|
Optional HTTP header name whose value is used as a tenant prefix in the cache key.
|
|
When set, keys are scoped as plano:affinity:{tenant_id}:{session_id}.
|
|
additionalProperties: false
|
|
routing_budget:
|
|
type: object
|
|
description: >
|
|
Per-session cost gate on model switching. Independent of prompt caching — it
|
|
applies whenever configured (presence of this block turns it on). The default
|
|
posture is to stick to the model a session is warm on. When routing proposes a
|
|
different model while that session's provider cache is plausibly still warm
|
|
(inferred from the idle gap vs. the provider's cache window), the actual
|
|
input-token cost of abandoning the cache —
|
|
context_tokens x (candidate_uncached_input_rate - anchor_cached_input_rate),
|
|
output cost deliberately excluded — accrues into the session's cumulative
|
|
switch spend. A paid switch is allowed only while that spend stays within
|
|
max_overhead_pct percent of the session's running never-switch baseline (what
|
|
staying on the anchor would have cost). An outright-cheaper switch is free but
|
|
never reduces the spend. Requires a cost source in model_metrics_sources.
|
|
properties:
|
|
max_overhead_pct:
|
|
type: number
|
|
minimum: 0
|
|
description: >
|
|
Cap on cumulative switching overhead, as a percentage of what the session
|
|
would have cost by never switching (a whole number: 20 = 20%). The promise
|
|
is "this conversation bills at most max_overhead_pct% above never-switching."
|
|
0 means never pay to switch (only outright-cheaper switches are allowed);
|
|
larger values buy more quality-driven switches. Typical range 10-30.
|
|
replenish_on_rebind:
|
|
type: boolean
|
|
description: "Reset the running baseline/spend totals when a cold session re-binds. Default true."
|
|
cache_read_discount:
|
|
type: number
|
|
minimum: 0
|
|
maximum: 1
|
|
description: >
|
|
Fallback used to estimate a model's cached input rate when the pricing feed
|
|
doesn't publish one (cached_rate = input_rate x discount). A pricing detail,
|
|
not a cost policy. Default 0.1.
|
|
record_counterfactual:
|
|
type: boolean
|
|
description: >
|
|
When true, a vetoed switch records the route the gate would have taken had
|
|
the switch been allowed, as the plano.switch.counterfactual_route span
|
|
attribute. Telemetry only — the counterfactual model is never dispatched.
|
|
Useful for evals/benchmarks. Default false.
|
|
required: ["max_overhead_pct"]
|
|
additionalProperties: false
|
|
additionalProperties: false
|
|
state_storage:
|
|
type: object
|
|
properties:
|
|
type:
|
|
type: string
|
|
enum:
|
|
- memory
|
|
- postgres
|
|
connection_string:
|
|
type: string
|
|
description: Required when type is postgres. Supports environment variable substitution using $VAR or ${VAR} syntax.
|
|
additionalProperties: false
|
|
required:
|
|
- type
|
|
# Note: connection_string is conditionally required based on type
|
|
# If type is 'postgres', connection_string must be provided
|
|
# If type is 'memory', connection_string is not needed
|
|
allOf:
|
|
- if:
|
|
properties:
|
|
type:
|
|
const: postgres
|
|
then:
|
|
required:
|
|
- connection_string
|
|
prompt_guards:
|
|
type: object
|
|
properties:
|
|
input_guards:
|
|
type: object
|
|
properties:
|
|
jailbreak:
|
|
type: object
|
|
properties:
|
|
on_exception:
|
|
type: object
|
|
properties:
|
|
message:
|
|
type: string
|
|
additionalProperties: false
|
|
required:
|
|
- message
|
|
additionalProperties: false
|
|
required:
|
|
- on_exception
|
|
additionalProperties: false
|
|
required:
|
|
- jailbreak
|
|
routing_preferences:
|
|
type: array
|
|
items:
|
|
type: object
|
|
properties:
|
|
name:
|
|
type: string
|
|
description:
|
|
type: string
|
|
models:
|
|
type: array
|
|
items:
|
|
type: string
|
|
minItems: 1
|
|
selection_policy:
|
|
type: object
|
|
properties:
|
|
prefer:
|
|
type: string
|
|
enum:
|
|
- cheapest
|
|
- fastest
|
|
- none
|
|
additionalProperties: false
|
|
required:
|
|
- prefer
|
|
additionalProperties: false
|
|
required:
|
|
- name
|
|
- description
|
|
- models
|
|
|
|
model_metrics_sources:
|
|
type: array
|
|
items:
|
|
oneOf:
|
|
- type: object
|
|
properties:
|
|
type:
|
|
type: string
|
|
const: cost
|
|
provider:
|
|
type: string
|
|
enum:
|
|
- digitalocean
|
|
- models.dev
|
|
url:
|
|
type: string
|
|
description: "Optional override for the pricing catalog endpoint. Defaults per provider (digitalocean: DO GenAI catalog; models.dev: https://models.dev/api.json)."
|
|
refresh_interval:
|
|
type: integer
|
|
minimum: 1
|
|
description: "Refresh interval in seconds"
|
|
model_aliases:
|
|
type: object
|
|
description: "Map catalog keys to Plano model names used in routing_preferences. DigitalOcean keys are 'lowercase(creator)/model_id'; models.dev keys are 'creator/model_id'. Example: 'openai/openai-gpt-oss-120b: openai/gpt-4o'"
|
|
additionalProperties:
|
|
type: string
|
|
required:
|
|
- type
|
|
- provider
|
|
additionalProperties: false
|
|
- type: object
|
|
properties:
|
|
type:
|
|
type: string
|
|
const: latency
|
|
provider:
|
|
type: string
|
|
enum:
|
|
- prometheus
|
|
url:
|
|
type: string
|
|
query:
|
|
type: string
|
|
refresh_interval:
|
|
type: integer
|
|
minimum: 1
|
|
description: "Refresh interval in seconds"
|
|
required:
|
|
- type
|
|
- provider
|
|
- url
|
|
- query
|
|
additionalProperties: false
|
|
|
|
additionalProperties: false
|
|
required:
|
|
- version
|
|
- listeners
|