plano/crates/llm_gateway
Troy Mitchell c34ff5b5fd feat: preserve original JSON bytes for prompt cache compatibility
Avoid re-serializing request bodies when unnecessary to maintain
JSON key order, whitespace, and unknown fields — critical for
prompt cache prefix matching on providers like Anthropic.

- routing_service: only re-serialize when routing_preferences
  were actually removed from the body
- stream_context: replace model name at byte level instead of
  full deserialization/re-serialization cycle
- Strip provider prefix from model name (e.g. 'custom-aws/claude-opus-4-6'
  -> 'claude-opus-4-6') before sending upstream

Signed-off-by: Troy Mitchell <i@troy-y.org>
2026-04-28 17:05:59 +08:00
..
src feat: preserve original JSON bytes for prompt cache compatibility 2026-04-28 17:05:59 +08:00
Cargo.toml Add support for Amazon Bedrock Converse and ConverseStream (#588) 2025-10-22 11:31:21 -07:00