plano/arch
Co Tran 80d2229053
Cotran/onnx conversion (#145)
* onnx replacement

* onnx conversion for nli and embedding model

* fix naming

* fix naming

* fix naming

* pin version
2024-10-08 14:37:48 -07:00
..
grafana rename envoyfilter => arch (#91) 2024-09-27 16:41:39 -07:00
prometheus rename envoyfilter => arch (#91) 2024-09-27 16:41:39 -07:00
src Cotran/onnx conversion (#145) 2024-10-08 14:37:48 -07:00
tests Hallucination integration with rust (#122) 2024-10-07 18:38:55 -07:00
tools ensure that we can call the new api.fc.archgw.com url, logging fixes … (#142) 2024-10-08 12:40:24 -07:00
arch_config_schema.yaml Add the ability to use LLM Providers from the Arch config (#112) 2024-10-03 10:57:01 -07:00
build_filter.sh rename envoyfilter => arch (#91) 2024-09-27 16:41:39 -07:00
Cargo.lock Implement Client trait for StreamContext (#134) 2024-10-07 16:50:15 -07:00
Cargo.toml Implement Client trait for StreamContext (#134) 2024-10-07 16:50:15 -07:00
docker-compose.dev.yaml Hallucination integration with rust (#122) 2024-10-07 18:38:55 -07:00
docker-compose.yaml model server build (#127) 2024-10-06 18:21:43 -07:00
Dockerfile Serialize tool calls for Arch FC (#131) 2024-10-07 00:03:25 -07:00
envoy.template.dev.yaml Hallucination integration with rust (#122) 2024-10-07 18:38:55 -07:00
envoy.template.yaml model server build (#127) 2024-10-06 18:21:43 -07:00
README.md rename envoyfilter => arch (#91) 2024-09-27 16:41:39 -07:00
requirements.txt simplify developer getting started experience (#102) 2024-10-01 10:02:23 -07:00
stage.env Salmanap/fix config generator (#124) 2024-10-05 10:49:47 -07:00

Envoy filter code for gateway

Add toolchain

$ rustup target add wasm32-wasi

Building

$ cargo build --target wasm32-wasi --release

Testing

$ cargo test

Using in Envoy

This example can be run with docker compose and has a matching Envoy configuration.

$ docker compose up

Examples

Direct response.

Send HTTP request to localhost:10000/hello:

$ curl localhost:10000/hello

Expected response:

HTTP/1.1 200 OK
content-length: 40
content-type: text/plain
custom-header: katanemo filter
date: Wed, 10 Jul 2024 16:59:43 GMT
server: envoy

Inline call.

Send HTTP request to localhost:10000/inline:

$ curl localhost:10000/hello
{
  "headers": {
    "Accept": "*/*",
    "Host": "localhost",
    "User-Agent": "curl/7.81.0",
    "X-Amzn-Trace-Id": "Root=1-637c4767-6e31776a0b407a0219b5b570",
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000"
  }
}

Expected Envoy logs:

[...] wasm log http_auth_random: Access granted.