nomyo-ai/nomyo-router
2
2
Fork 0
Code Issues 1 Pull requests Projects Releases 5 Packages 1 Wiki Activity Actions
A transparent (O)llama proxy with model deployment aware routing which auto-manages multiple (O)llama instances in a given network.
6 commits 3 branches 5 tags 1.2 MiB
Find a file
alpha-nerd-nomyo 019490b6bd
Update README.md
2025-08-26 18:42:32 +02:00
config.yaml Initial commit 2025-08-26 18:19:43 +02:00
LICENSE Create LICENSE 2025-08-26 18:14:45 +02:00
README.md Update README.md 2025-08-26 18:42:32 +02:00
requirements.txt Initial commit 2025-08-26 18:19:43 +02:00
router.py Initial commit 2025-08-26 18:19:43 +02:00

README.md

NOMYO Router

is a transparent proxy for Ollama with model deployment aware routing.

It runs between your frontend application and Ollama backend and is transparent for both, the front- and backend.

arch

Installation

Copy/Clone the repository, edit the config.yaml by adding your Ollama backend servers and the max_concurrent_connections setting per endpoint. This equals to your OLLAMA_NUM_PARALLEL config settings.

Run the NOMYO Router in a dedicated virtual environment, install the requirements and run with uvicorn:

python3 -m venv .venv/router
source .venv/router/bin/activate
pip3 install requirements.txt -r

finally you can

uvicorn router:app --host 127.0.0.1 --port 12434

Routing

routing