mirror of
https://github.com/katanemo/plano.git
synced 2026-06-08 14:55:14 +02:00
a58a283e20
Add retry policy configuration types to support automatic retry and failover for LLM requests: - RetryPolicy: top-level config with fallback_models, default_strategy, default_max_attempts, and per-status-code overrides - BackoffConfig: exponential backoff with base_ms, max_ms, jitter, and scope (per-model, per-provider, or global) - RetryAfterConfig: Retry-After header handling with block scope and duration limits - HighLatencyConfig: latency-based blocking with threshold, measurement type, and trigger conditions - LatencyTriggerConfig: min_triggers and trigger_window for debouncing - RetryStrategy enum: same_model, same_provider, different_provider - StatusCodeEntry: flexible status code matching (single, range, list) Also add retry_policy field to GatewayConfig with Default impl. Signed-off-by: Troy Mitchell <i@troy-y.org> |
||
|---|---|---|
| .. | ||
| .vscode | ||
| brightstaff | ||
| common | ||
| hermesllm | ||
| llm_gateway | ||
| prompt_gateway | ||
| build.sh | ||
| Cargo.lock | ||
| Cargo.toml | ||