plano/docs/DATA_CONTRACTS.md

# Data Contracts — Inter-Component Communication

This document defines the contracts between Plano's components: custom HTTP headers, internal API formats, streaming protocols, and Envoy routing conventions. Breaking any of these contracts will cause silent routing failures.

---

## 1. Custom Header Protocol

All custom headers are defined in `common/src/consts.rs`. This is the **single source of truth** — if a header name appears in `envoy.template.yaml` or Brightstaff code, it must match the constant in `consts.rs`.

### Routing Headers (Envoy-critical)

These headers are used in Envoy's `route_config` for cluster selection. Changing them requires updating `envoy.template.yaml`.

| Header | Constant | Set By | Read By | Value Format | Purpose |
|---|---|---|---|---|---|
| `x-arch-llm-provider` | `ARCH_ROUTING_HEADER` | WASM filters | Envoy routes | Provider slug (e.g., `openai`, `anthropic`) | Selects the LLM provider cluster in Envoy |
| `x-arch-upstream` | `ARCH_UPSTREAM_HOST_HEADER` | WASM filters, Brightstaff | Envoy routes | Cluster name (e.g., agent endpoint name) | Routes to a specific upstream cluster |
| `x-arch-llm-provider-hint` | `ARCH_PROVIDER_HINT_HEADER` | Brightstaff | llm_gateway | `provider/model` (e.g., `openai/gpt-4`) | Hints which provider+model to use |
| `x-arch-agent-listener-name` | — | Envoy (set in route config) | Brightstaff | Listener name string | Identifies which agent listener a request arrived on |

### Internal State Headers (WASM filter internal)

These headers pass state between the prompt_gateway filter's request/response phases or between prompt_gateway and the function calling service.

| Header | Constant | Set By | Read By | Value Format | Purpose |
|---|---|---|---|---|---|
| `x-arch-state` | `X_ARCH_STATE_HEADER` | prompt_gateway | prompt_gateway | Base64-encoded JSON (`ArchState`) | Multi-turn conversation state across filter invocations |
| `x-arch-tool-call-message` | `X_ARCH_TOOL_CALL` | prompt_gateway | prompt_gateway | JSON string | Tool call metadata for API orchestration |
| `x-arch-api-response-message` | `X_ARCH_API_RESPONSE` | prompt_gateway | prompt_gateway | JSON string | Developer API response data |
| `x-arch-fc-model-response` | `X_ARCH_FC_MODEL_RESPONSE` | prompt_gateway | prompt_gateway | JSON string | Raw Arch-Function model response |
| `x-arch-llm-route` | `LLM_ROUTE_HEADER` | Brightstaff | llm_gateway | Route name string | LLM route decision result |

### Signaling Headers

| Header | Constant | Set By | Read By | Purpose |
|---|---|---|---|---|
| `x-arch-streaming-request` | `ARCH_IS_STREAMING_HEADER` | Brightstaff | llm_gateway | Indicates the request is streaming mode |
| `x-arch-ratelimit-selector` | `RATELIMIT_SELECTOR_HEADER_KEY` | Client / Envoy | llm_gateway | Key for per-tenant rate limit partitioning |

### Standard Headers Used

| Header | Constant | Purpose |
|---|---|---|
| `x-request-id` | `REQUEST_ID_HEADER` | Request tracing (set by Envoy or caller) |
| `x-envoy-original-path` | `ENVOY_ORIGINAL_PATH_HEADER` | Original path before Envoy rewrites |
| `x-envoy-max-retries` | `ENVOY_RETRY_HEADER` | Retry count for Envoy's retry policy |
| `traceparent` | `TRACE_PARENT_HEADER` | W3C Trace Context for OpenTelemetry |

---

## 2. Internal Cluster Names

Defined in `consts.rs` and referenced in `envoy.template.yaml`:

| Constant | Value | Target | Purpose |
|---|---|---|---|
| `MODEL_SERVER_NAME` | `"bright_staff"` | localhost:9091 | Brightstaff service |
| `ARCH_INTERNAL_CLUSTER_NAME` | `"arch_internal"` | localhost:11000 | Outbound API router |
| `ARCH_FC_CLUSTER` | `"arch"` | archfc.katanemo.dev:443 | Katanemo Arch-Function model |

Additional clusters generated from config:
- `arch_prompt_gateway_listener` → localhost:10001
- `arch_listener_llm` → localhost:12001
- Per-provider clusters (e.g., `openai`, `anthropic`, `gemini`) from `envoy.template.yaml`
- Per-agent/endpoint clusters from user config

---

## 3. Internal API Formats

### Brightstaff → Envoy (LLM requests via :12001)

Brightstaff sends OpenAI-compatible `ChatCompletionsRequest` JSON to `localhost:12001` with:
- `x-arch-llm-provider-hint: <provider>/<model>` to select the provider
- `x-arch-is-streaming: true/false` to indicate streaming
- Standard `Content-Type: application/json`
- `traceparent` for distributed tracing

The `llm_gateway` WASM filter at :12001 transforms the request to the target provider's format.

### Brightstaff → Envoy (Agent/API requests via :11000)

Brightstaff sends requests to `localhost:11000` with:
- `x-arch-upstream-host: <cluster_name>` to route to the target agent/API
- `x-envoy-max-retries: 3` for resilience

**MCP Agent Protocol:**
```
POST /  (with x-arch-upstream-host)
Content-Type: application/json

# Step 1: Initialize
{"jsonrpc":"2.0","method":"initialize","id":"<uuid>","params":{...}}

# Step 2: Initialized notification
{"jsonrpc":"2.0","method":"notifications/initialized"}

# Step 3: Tool call
{"jsonrpc":"2.0","method":"tools/call","id":"<uuid>","params":{"name":"<tool>","arguments":{...}}}
```

**HTTP Agent Protocol:**
```
POST /  (with x-arch-upstream-host)
Content-Type: application/json

[{"role":"user","content":"..."},{"role":"assistant","content":"..."}]
```
Response: Array of messages.

### prompt_gateway → Arch-Function (/function_calling)

```
POST /function_calling
Content-Type: application/json

{
  "messages": [...],
  "tools": [...],
  "model": "Arch-Function",
  "stream": false,
  "metadata": {"raw_response": true, "logprobs": true}
}
```

Response contains `tool_calls`, `response`, or `clarification` in the assistant message content (JSON string).

---

## 4. Streaming Protocol

### SSE (Server-Sent Events) — Standard LLM Streaming

All streaming LLM responses use SSE format:
```
data: {"id":"...","choices":[...]}\n\n
data: {"id":"...","choices":[...]}\n\n
data: [DONE]\n\n
```

**Important:** SSE events can be split across HTTP chunks. The `llm_gateway` uses `SseStreamBuffer` and `SseChunkProcessor` (from `hermesllm`) to reassemble events across chunk boundaries before processing.

### Bedrock Binary Streaming

Amazon Bedrock uses AWS Event Stream binary protocol instead of SSE. The `BedrockBinaryFrameDecoder` in `hermesllm` handles decoding.

### Brightstaff Streaming

Brightstaff uses `tokio::sync::mpsc` channels to stream responses:
1. Spawns a background task to read from upstream (via `reqwest`)
2. Parses SSE events, optionally transforms them
3. Sends chunks through the mpsc channel
4. Axum's `StreamBody` delivers to the client

---

## 5. Configuration Injection

### WASM Filter Configuration

Envoy injects config into WASM filters via the `configuration` field in the filter definition:

- **prompt_gateway** receives: `prompt_targets`, `prompt_guards`, `system_prompt`, `endpoints`, `overrides`, `tracing`
- **llm_gateway** receives: `model_providers`, `ratelimits`, `overrides`

Both receive YAML strings parsed by `serde_yaml` in each filter's `RootContext::on_configure()`.

### Brightstaff Configuration

Brightstaff reads `arch_config_rendered.yaml` (path from `ARCH_CONFIG_PATH_RENDERED` env var), which contains the full rendered config including `model_providers`, `agents`, `filters`, `listeners`, `routing`, `model_aliases`, `state_storage`, `tracing`, and `overrides`.

---

## 6. Timeouts

All timeouts are defined in `consts.rs`:

| Constant | Value | Used For |
|---|---|---|
| `ARCH_FC_REQUEST_TIMEOUT_MS` | 30,000 ms | Arch-Function model calls from prompt_gateway |
| `DEFAULT_TARGET_REQUEST_TIMEOUT_MS` | 30,000 ms | Default prompt target endpoint calls |
| `API_REQUEST_TIMEOUT_MS` | 30,000 ms | Developer API calls from prompt_gateway |
| `MODEL_SERVER_REQUEST_TIMEOUT_MS` | 30,000 ms | Model server calls |

Envoy also enforces its own route-level timeouts configured in `envoy.template.yaml` (default 300s for LLM routes).

---

## 7. Error Response Format

All error responses from Brightstaff follow this format:

```json
{
  "error": {
    "message": "Human-readable error description",
    "type": "error_type",
    "code": 400
  }
}
```

The `llm_gateway` WASM filter returns errors as:
- HTTP 429 for rate limit exceeded
- HTTP 503 for provider unavailable
- The original upstream error status code for pass-through errors

---

## 8. Contract Change Checklist

When modifying any data contract:

- [ ] Update the constant in `common/src/consts.rs`
- [ ] Grep the entire codebase for the old value (`grep -r "old_value" crates/`)
- [ ] Update `config/envoy.template.yaml` if the header is used in routing
- [ ] Update `cli/planoai/config_generator.py` if the config schema changed
- [ ] Update `config/arch_config_schema.yaml` if user-facing config changed
- [ ] Run `cargo test --workspace` to catch compile/test failures
- [ ] Run `cd cli && python -m pytest test/` for config generation tests