mirror of
https://github.com/katanemo/plano.git
synced 2026-06-05 14:45:15 +02:00
* add toxic/jailbreak model * fix path loading model * fix syntax * fix bug,lint, format * fix bug * formatting * add parallel + chunking * fix bug * working version * fix onnnx name erorr * device * fix jailbreak config * fix syntax error * format * add requirement + cli download for dockerfile * add task * add skeleton change for envoy filter for prompt guard * fix hardware config * fix bug * add config changes * add gitignore * merge main * integrate arch-guard with filter * add hardware config * nothing * add hardware config feature * fix requirement * fix chat ui * fix onnx * fix lint * remove non intel cpu * remove onnx * working version * modify docker * fix guard time * add nvidia support * remove nvidia * add gpu * add gpu * add gpu support * add gpu support for compose * add gpu support for compose * add gpu support for compose * add gpu support for compose * add gpu support for compose * fix docker file * fix int test * correct gpu docker * upgrad python 10 * fix logits to be gpu compatible * default to cpu dockerfile * resolve comments * fix lint + unused parameters * fix * remove eetq install for cpu * remove deploy gpu --------- Co-authored-by: Adil Hafeez <adil@katanemo.com> |
||
|---|---|---|
| .. | ||
| grafana | ||
| prometheus | ||
| src | ||
| tests | ||
| build_filter.sh | ||
| Cargo.lock | ||
| Cargo.toml | ||
| docker-compose.yaml | ||
| Dockerfile | ||
| download_mistral_7b.sh | ||
| envoy.template.yaml | ||
| init_vector_store.sh | ||
| katanemo-config.yaml | ||
| README.md | ||
Envoy filter code for gateway
Add toolchain
$ rustup target add wasm32-wasi
Building
$ cargo build --target wasm32-wasi --release
Testing
$ cargo test
Using in Envoy
This example can be run with docker compose
and has a matching Envoy configuration.
$ docker compose up
Examples
Direct response.
Send HTTP request to localhost:10000/hello:
$ curl localhost:10000/hello
Expected response:
HTTP/1.1 200 OK
content-length: 40
content-type: text/plain
custom-header: katanemo filter
date: Wed, 10 Jul 2024 16:59:43 GMT
server: envoy
Inline call.
Send HTTP request to localhost:10000/inline:
$ curl localhost:10000/hello
{
"headers": {
"Accept": "*/*",
"Host": "localhost",
"User-Agent": "curl/7.81.0",
"X-Amzn-Trace-Id": "Root=1-637c4767-6e31776a0b407a0219b5b570",
"X-Envoy-Expected-Rq-Timeout-Ms": "15000"
}
}
Expected Envoy logs:
[...] wasm log http_auth_random: Access granted.