Document model_providers headers in configuration reference (#950)

* Document model_providers headers in configuration reference

Co-authored-by: Musa <musa@spherrrical.dev>

* ci: retrigger workflow

Co-authored-by: Musa <musa@spherrrical.dev>

* fix(llm_gateway): buffer non-streaming response body until end_of_stream

Wait for the full upstream body before JSON parsing to avoid truncated
responses on chunked replies. Retry currency_exchange demo tests on flake.

Co-authored-by: Musa <musa@spherrrical.dev>

* fix(llm_gateway): read full non-streaming body when final chunk is empty

Co-authored-by: Musa <musa@spherrrical.dev>

* fix(llm_gateway): read full non-streaming body with usize::MAX at end_of_stream

Co-authored-by: Musa <musa@spherrrical.dev>

* fix(llm_gateway): use envoy body_size for response body replacement

Co-authored-by: Musa <musa@spherrrical.dev>

* docs: explain model_providers headers in configuration reference

Revert unrelated llm_gateway and demo test runner changes.

Co-authored-by: Musa <musa@spherrrical.dev>

* chore: drop unrelated changes, keep docs-only diff

Co-authored-by: Musa <musa@spherrrical.dev>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
This commit is contained in:
Musa 2026-06-03 13:38:39 -07:00 committed by GitHub
parent f3d6ea41ad
commit b5ebb1beea
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 55 additions and 0 deletions

View file

@ -7,6 +7,29 @@ The following is a complete reference of the ``plano_config.yml`` that controls
the Plano gateway. This where you enable capabilities like routing to upstream LLm providers, defining prompt_targets
where prompts get routed to, apply guardrails, and enable critical agent observability features.
Model provider headers
----------------------
Each entry under ``model_providers`` (or the legacy ``llm_providers`` alias) may include a ``headers`` map of extra
HTTP headers that Plano adds to upstream LLM requests. Plano applies these headers after it sets authentication from
``access_key`` or ``passthrough_auth``, so you can supply provider-specific metadata without replacing the configured
credentials.
- **Type:** map of strings (header name → value)
- **Optional:** yes
- **Common uses:** required ``User-Agent`` values, organization or account identifiers, or other headers some APIs expect
.. code-block:: yaml
model_providers:
- model: moonshotai/kimi-for-coding
access_key: $MOONSHOTAI_API_KEY
base_url: https://api.kimi.com/coding/v1
headers:
User-Agent: "KimiCLI/1.3"
The example below includes this and other provider options in context.
.. literalinclude:: includes/plano_config_full_reference.yaml
:language: yaml
:linenos:

View file

@ -47,6 +47,14 @@ model_providers:
http_host: api.custom-provider.com
access_key: $CUSTOM_API_KEY
# headers: optional map of extra HTTP headers sent on upstream requests (after auth).
# Use for provider-specific requirements such as User-Agent, org IDs, or account headers.
- model: moonshotai/kimi-for-coding
access_key: $MOONSHOTAI_API_KEY
base_url: https://api.kimi.com/coding/v1
headers:
User-Agent: "KimiCLI/1.3"
# Model aliases - use friendly names instead of full provider model names
model_aliases:
fast-llm:

View file

@ -88,6 +88,18 @@ listeners:
port: 443
protocol: https
provider_interface: openai
- access_key: $MOONSHOTAI_API_KEY
base_url: https://api.kimi.com/coding/v1
base_url_path_prefix: /coding/v1
cluster_name: moonshotai_api.kimi.com
endpoint: api.kimi.com
headers:
User-Agent: KimiCLI/1.3
model: kimi-for-coding
name: moonshotai/kimi-for-coding
port: 443
protocol: https
provider_interface: moonshotai
name: model_1
output_filters:
- input_guards
@ -144,6 +156,18 @@ model_providers:
port: 443
protocol: https
provider_interface: openai
- access_key: $MOONSHOTAI_API_KEY
base_url: https://api.kimi.com/coding/v1
base_url_path_prefix: /coding/v1
cluster_name: moonshotai_api.kimi.com
endpoint: api.kimi.com
headers:
User-Agent: KimiCLI/1.3
model: kimi-for-coding
name: moonshotai/kimi-for-coding
port: 443
protocol: https
provider_interface: moonshotai
- internal: true
model: Plano-Orchestrator
name: plano-orchestrator