Update README.md
This commit is contained in:
parent
cb7224c030
commit
e06745c39e
1 changed files with 11 additions and 0 deletions
11
README.md
11
README.md
|
|
@ -10,6 +10,17 @@ It runs between your frontend application and Ollama backend and is transparent
|
|||
|
||||
Copy/Clone the repository, edit the config.yaml by adding your Ollama backend servers and the max_concurrent_connections setting per endpoint. This equals to your OLLAMA_NUM_PARALLEL config settings.
|
||||
|
||||
```
|
||||
# config.yaml
|
||||
endpoints:
|
||||
- http://ollama0:11434
|
||||
- http://ollama1:11434
|
||||
- http://ollama2:11434
|
||||
|
||||
# Maximum concurrent connections *per endpoint‑model pair*
|
||||
max_concurrent_connections: 2
|
||||
```
|
||||
|
||||
Run the NOMYO Router in a dedicated virtual environment, install the requirements and run with uvicorn:
|
||||
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue