plano/crates/llm_gateway/src
Troy Mitchell c34ff5b5fd feat: preserve original JSON bytes for prompt cache compatibility
Avoid re-serializing request bodies when unnecessary to maintain
JSON key order, whitespace, and unknown fields — critical for
prompt cache prefix matching on providers like Anthropic.

- routing_service: only re-serialize when routing_preferences
  were actually removed from the body
- stream_context: replace model name at byte level instead of
  full deserialization/re-serialization cycle
- Strip provider prefix from model name (e.g. 'custom-aws/claude-opus-4-6'
  -> 'claude-opus-4-6') before sending upstream

Signed-off-by: Troy Mitchell <i@troy-y.org>
2026-04-28 17:05:59 +08:00
..
filter_context.rs use standard tracing and logging in brightstaff (#721) 2026-02-09 13:33:27 -08:00
lib.rs Break apart metrics into their own module (#335) 2024-12-09 10:46:46 -08:00
metrics.rs Break apart metrics into their own module (#335) 2024-12-09 10:46:46 -08:00
stream_context.rs feat: preserve original JSON bytes for prompt cache compatibility 2026-04-28 17:05:59 +08:00