mirror of
https://github.com/katanemo/plano.git
synced 2026-05-21 13:55:15 +02:00
rename filter_chain/output_filter_chain to input_filters/output_filters, scope output filters to chat completions only
This commit is contained in:
parent
e458daf162
commit
ca64833686
13 changed files with 69 additions and 60 deletions
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Run content-safety filters on direct LLM requests — no agent layer required.
|
||||
|
||||
This demo uses the `filter_chain` feature on a **model-type listener** to intercept
|
||||
This demo uses the `input_filters` feature on a **model-type listener** to intercept
|
||||
`/v1/chat/completions` requests and block unsafe content before they reach the LLM provider.
|
||||
|
||||
## Architecture
|
||||
|
|
@ -10,7 +10,7 @@ This demo uses the `filter_chain` feature on a **model-type listener** to interc
|
|||
```
|
||||
Client ──► Plano (model listener :12000)
|
||||
│
|
||||
├─ filter_chain: content_guard ──► Block / Allow
|
||||
├─ input_filters: content_guard ──► Block / Allow
|
||||
│
|
||||
└─ model_provider: openai/gpt-4o-mini
|
||||
```
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ listeners:
|
|||
- type: model
|
||||
name: llm_gateway
|
||||
port: 12000
|
||||
filter_chain:
|
||||
input_filters:
|
||||
- content_guard
|
||||
|
||||
tracing:
|
||||
|
|
|
|||
|
|
@ -2,20 +2,20 @@
|
|||
|
||||
Automatically redact PII from LLM requests and restore it in responses — inspired by [Uber's GenAI Gateway PII Redactor](https://www.uber.com/blog/genai-gateway/).
|
||||
|
||||
This demo uses both `filter_chain` (input) and `output_filter_chain` (output) on a **model-type listener** to anonymize PII before it reaches the LLM provider, then de-anonymize the response before returning it to the client.
|
||||
This demo uses both `input_filters` and `output_filters` on a **model-type listener** to anonymize PII before it reaches the LLM provider, then de-anonymize the response before returning it to the client.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Client ──► Plano (model listener :12000)
|
||||
│
|
||||
├─ filter_chain: pii_anonymizer
|
||||
├─ input_filters: pii_anonymizer
|
||||
│ └─ Replace PII with [EMAIL_0], [SSN_0], etc.
|
||||
│
|
||||
├─ model_provider: openai/gpt-4o-mini
|
||||
│ └─ LLM only sees anonymized data
|
||||
│
|
||||
└─ output_filter_chain: pii_deanonymizer
|
||||
└─ output_filters: pii_deanonymizer
|
||||
└─ Restore [EMAIL_0] → original email (per-chunk for streaming)
|
||||
```
|
||||
|
||||
|
|
@ -82,7 +82,7 @@ Check the PII filter service logs in the terminal running `start_agents.sh`. You
|
|||
|
||||
## How Streaming De-anonymization Works
|
||||
|
||||
For streaming responses, each SSE chunk is sent through the output filter chain as it arrives from the LLM:
|
||||
For streaming responses, each SSE chunk is sent through the output filters as it arrives from the LLM:
|
||||
|
||||
1. Plano receives a chunk with content like `"The email [EMAIL_0] belongs to..."`
|
||||
2. The chunk content is sent to the `/deanonymize` endpoint
|
||||
|
|
|
|||
|
|
@ -17,9 +17,9 @@ listeners:
|
|||
- type: model
|
||||
name: llm_gateway
|
||||
port: 12000
|
||||
filter_chain:
|
||||
input_filters:
|
||||
- pii_anonymizer
|
||||
output_filter_chain:
|
||||
output_filters:
|
||||
- pii_deanonymizer
|
||||
|
||||
tracing:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue