diff --git a/README.md b/README.md
index d7e08c1..1780c40 100644
--- a/README.md
+++ b/README.md
@@ -14,12 +14,17 @@ Copy/Clone the repository, edit the config.yaml by adding your Ollama backend se
```
# config.yaml
+# Ollama or OpenAI API V1 endpoints
endpoints:
- http://ollama0:11434
- http://ollama1:11434
- http://ollama2:11434
- https://api.openai.com/v1
+# llama.cpp server endpoints
+llama_server_endpoints:
+ - http://192.168.0.33:8889/v1
+
# Maximum concurrent connections *per endpoint‑model pair*
max_concurrent_connections: 2
@@ -34,6 +39,7 @@ api_keys:
"http://192.168.0.51:11434": "ollama"
"http://192.168.0.52:11434": "ollama"
"https://api.openai.com/v1": "${OPENAI_KEY}"
+ "http://192.168.0.33:8889/v1": "llama"
```
Run the NOMYO Router in a dedicated virtual environment, install the requirements and run with uvicorn:
@@ -58,6 +64,12 @@ finally you can
uvicorn router:app --host 127.0.0.1 --port 12434
```
+in very high concurrent scenarios (> 500 simultaneous requests) you can also run with uvloop
+
+```
+uvicorn router:app --host 127.0.0.1 --port 12434 --loop uvloop
+```
+
## Docker Deployment
Build the container image locally:
@@ -98,7 +110,6 @@ This way the Ollama backend servers are utilized more efficient than by simply u
NOMYO Router also supports OpenAI API compatible v1 backend servers.
-
## Supplying the router API key
If you set `nomyo-router-api-key` in `config.yaml` (or `NOMYO_ROUTER_API_KEY` env), every request to NOMYO Router must include the key:
@@ -107,6 +118,7 @@ If you set `nomyo-router-api-key` in `config.yaml` (or `NOMYO_ROUTER_API_KEY` en
- Query param (fallback): `?api_key=`
Examples:
+
```bash
curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags
curl "http://localhost:12434/api/tags?api_key=$NOMYO_ROUTER_API_KEY"
diff --git a/requirements.txt b/requirements.txt
index aa51a0f..f1db419 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -36,5 +36,6 @@ tqdm==4.67.1
typing-inspection==0.4.1
typing_extensions==4.14.1
uvicorn==0.38.0
+uvloop
yarl==1.20.1
aiosqlite