add: Optional router-level API key that gates router/API/web UI access

Optional router-level API key that gates router/API/web UI access (leave empty to disable)

## Supplying the router API key

If you set `nomyo-router-api-key` in `config.yaml` (or `NOMYO_ROUTER_API_KEY` env), every request to NOMYO Router must include the key:

- HTTP header (recommended): `Authorization: Bearer <router_key>`
- Query param (fallback): `?api_key=<router_key>`

Examples:
```bash
curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags
curl "http://localhost:12434/api/tags?api_key=$NOMYO_ROUTER_API_KEY"
```
This commit is contained in:
YetheSamartaka 2026-01-14 09:28:02 +01:00
parent 6828411f95
commit eca4a92a33
9 changed files with 412 additions and 25 deletions

View file

@ -23,6 +23,9 @@ endpoints:
# Maximum concurrent connections *per endpointmodel pair*
max_concurrent_connections: 2
# Optional router-level API key to lock down router + dashboard (leave empty to disable)
nomyo-router-api-key: ""
# API keys for remote endpoints
# Set an environment variable like OPENAI_KEY
# Confirm endpoints are exactly as in endpoints block
@ -45,6 +48,8 @@ pip3 install -r requirements.txt
```
export OPENAI_KEY=YOUR_SECRET_API_KEY
# Optional: router-level key (clients must send Authorization: Bearer)
# export NOMYO_ROUTER_API_KEY=YOUR_ROUTER_KEY
```
finally you can
@ -91,3 +96,17 @@ This way the Ollama backend servers are utilized more efficient than by simply u
![routing](https://github.com/user-attachments/assets/ed05dfbb-fcc8-4ff2-b8ca-3cdce2660c9f)
NOMYO Router also supports OpenAI API compatible v1 backend servers.
## Supplying the router API key
If you set `nomyo-router-api-key` in `config.yaml` (or `NOMYO_ROUTER_API_KEY` env), every request to NOMYO Router must include the key:
- HTTP header (recommended): `Authorization: Bearer <router_key>`
- Query param (fallback): `?api_key=<router_key>`
Examples:
```bash
curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags
curl "http://localhost:12434/api/tags?api_key=$NOMYO_ROUTER_API_KEY"
```

View file

@ -8,6 +8,9 @@ endpoints:
# Maximum concurrent connections *per endpointmodel pair* (equals to OLLAMA_NUM_PARALLEL)
max_concurrent_connections: 2
# Optional router-level API key that gates router/API/web UI access (leave empty to disable)
nomyo-router-api-key: ""
# API keys for remote endpoints
# Set an environment variable like OPENAI_KEY
# Confirm endpoints are exactly as in endpoints block

View file

@ -36,6 +36,8 @@ doc/
endpoints:
- http://localhost:11434
max_concurrent_connections: 2
# Optional router-level API key (leave blank to disable)
nomyo-router-api-key: ""
```
3. **Run the router**:
@ -135,3 +137,15 @@ For additional help:
5. **Scale as needed** by adding more endpoints
Happy routing! 🚀
## Router API key usage
If the router API key is set (`NOMYO_ROUTER_API_KEY` env or `nomyo-router-api-key` in config), include it in every request:
- Header (preferred): Authorization: Bearer <router_key>
- Query param: ?api_key=<router_key>
Example:
```bash
curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags
```

View file

@ -14,6 +14,9 @@ endpoints:
# Maximum concurrent connections *per endpointmodel pair*
max_concurrent_connections: 2
# Optional router-level API key to secure the router and dashboard (leave blank to disable)
nomyo-router-api-key: ""
```
### Complete Example
@ -29,6 +32,9 @@ endpoints:
# Maximum concurrent connections *per endpointmodel pair* (equals to OLLAMA_NUM_PARALLEL)
max_concurrent_connections: 2
# Optional router-level API key to secure the router and dashboard (leave blank to disable)
nomyo-router-api-key: ""
# API keys for remote endpoints
# Set an environment variable like OPENAI_KEY
# Confirm endpoints are exactly as in endpoints block
@ -80,6 +86,21 @@ max_concurrent_connections: 4
- When this limit is reached, the router will route requests to other endpoints with available capacity
- Higher values allow more parallel requests but may increase memory usage
### `router_api_key`
**Type**: `str` (optional)
**Description**: Shared secret that gates access to the NOMYO Router APIs and dashboard. When set, clients must send `Authorization: Bearer <key>` or an `api_key` query parameter.
**Example**:
```yaml
nomyo-router-api-key: "super-secret-value"
```
**Notes**:
- Leave this blank or omit it to disable router-level authentication.
- You can also set the `NOMYO_ROUTER_API_KEY` environment variable to avoid storing the key in plain text.
### `api_keys`
**Type**: `dict[str, str]`
@ -118,6 +139,15 @@ export NOMYO_ROUTER_CONFIG_PATH=/etc/nomyo-router/config.yaml
export NOMYO_ROUTER_DB_PATH=/var/lib/nomyo-router/token_counts.db
```
### `NOMYO_ROUTER_API_KEY`
**Description**: Router-level API key. When set, all router endpoints and the dashboard require this key via `Authorization: Bearer <key>` or the `api_key` query parameter.
**Example**:
```bash
export NOMYO_ROUTER_API_KEY=your_router_api_key
```
### API-Specific Keys
You can set API keys directly as environment variables:
@ -195,3 +225,15 @@ The configuration is loaded at startup and cannot be changed without restarting
## Example Configurations
See the [examples](examples/) directory for ready-to-use configuration examples.
### Using the router API key
When `router_api_key`/`NOMYO_ROUTER_API_KEY` is set, clients must send it on every request:
- Header (recommended): Authorization: Bearer <router_key>
- Query param (fallback): ?api_key=<router_key>
Example:
```bash
curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags
```

View file

@ -217,6 +217,7 @@ data:
endpoints:
- http://ollama-service:11434
max_concurrent_connections: 2
nomyo-router-api-key: ""
---
apiVersion: v1
kind: PersistentVolumeClaim
@ -283,6 +284,9 @@ endpoints:
max_concurrent_connections: 4
# Optional router-level API key to secure the router and dashboard (leave blank to disable)
nomyo-router-api-key: ""
api_keys:
"https://api.openai.com/v1": "${OPENAI_KEY}"
```

View file

@ -6,6 +6,9 @@ endpoints:
max_concurrent_connections: 2
# Optional router-level API key to secure the router and dashboard (leave blank to disable)
nomyo-router-api-key: ""
# Multi-endpoint configuration with local Ollama instances
# endpoints:
# - http://ollama-worker1:11434

View file

@ -21,6 +21,9 @@ endpoints:
- http://localhost:11434
max_concurrent_connections: 2
# Optional router-level API key (leave blank to disable)
nomyo-router-api-key: ""
```
### 3. Run the Router
@ -346,3 +349,15 @@ print(f"Available models: {[m.id for m in response.data]}")
## Examples
See the [examples](examples/) directory for complete integration examples.
### Authentication to NOMYO Router
If a router API key is configured, include it with each request:
- Header: Authorization: Bearer <router_key>
- Query: ?api_key=<router_key>
Example (tags):
```bash
curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags
```

113
router.py
View file

@ -6,14 +6,14 @@ version: 0.5
license: AGPL
"""
# -------------------------------------------------------------
import orjson, time, asyncio, yaml, ollama, openai, os, re, aiohttp, ssl, random, base64, io, enhance
import orjson, time, asyncio, yaml, ollama, openai, os, re, aiohttp, ssl, random, base64, io, enhance, secrets
from datetime import datetime, timezone
from pathlib import Path
# Directory containing static files (relative to this script)
STATIC_DIR = Path(__file__).parent / "static"
from typing import Dict, Set, List, Optional
from urllib.parse import urlparse
from urllib.parse import urlparse, parse_qsl, urlencode
from fastapi import FastAPI, Request, HTTPException
from fastapi_sse import sse_handler
from fastapi.staticfiles import StaticFiles
@ -41,6 +41,20 @@ _subscribers: Set[asyncio.Queue] = set()
_subscribers_lock = asyncio.Lock()
token_queue: asyncio.Queue[tuple[str, str, int, int]] = asyncio.Queue()
# -------------------------------------------------------------
# Secret handling
# -------------------------------------------------------------
def _mask_secrets(text: str) -> str:
"""
Mask common API key patterns to avoid leaking secrets in logs or error payloads.
"""
if not text:
return text
# OpenAI-style keys (sk-...) and generic "api key" mentions
text = re.sub(r"sk-[A-Za-z0-9]{4}[A-Za-z0-9_-]*", "sk-***redacted***", text)
text = re.sub(r"(?i)(api[-_ ]key\\s*[:=]\\s*)([^\\s]+)", r"\\1***redacted***", text)
return text
# ------------------------------------------------------------------
# Globals
# ------------------------------------------------------------------
@ -78,6 +92,8 @@ class Config(BaseSettings):
max_concurrent_connections: int = 1
api_keys: Dict[str, str] = Field(default_factory=dict)
# Optional router-level API key used to gate access to this service and dashboard
router_api_key: Optional[str] = Field(default=None, env="NOMYO_ROUTER_API_KEY")
# Database configuration
db_path: str = Field(default=os.getenv("NOMYO_ROUTER_DB_PATH", "token_counts.db"))
@ -108,6 +124,29 @@ class Config(BaseSettings):
with path.open("r", encoding="utf-8") as fp:
data = yaml.safe_load(fp) or {}
cleaned = cls._expand_env_refs(data)
if isinstance(cleaned, dict):
# Accept hyphenated config key and map it to the field name
key_aliases = [
# canonical field name
"router_api_key",
# lowercase, hyphen/underscore variants
"nomyo-router-api-key",
"nomyo_router_api_key",
"nomyo-router_api_key",
"nomyo_router-api_key",
# uppercase env-style variants
"NOMYO-ROUTER_API_KEY",
"NOMYO_ROUTER_API_KEY",
]
for alias in key_aliases:
if alias in cleaned:
cleaned["router_api_key"] = cleaned.get("router_api_key", cleaned.pop(alias))
break
# If not present in YAML (or empty), fall back to env var explicitly
if not cleaned.get("router_api_key"):
env_key = os.getenv("NOMYO_ROUTER_API_KEY")
if env_key:
cleaned["router_api_key"] = env_key
return cls(**cleaned)
return cls()
@ -145,6 +184,69 @@ default_headers={
"X-Title": "NOMYO Router",
}
# -------------------------------------------------------------
# Router-level authentication (optional)
# -------------------------------------------------------------
def _extract_router_api_key(request: Request) -> Optional[str]:
"""
Extract the provided router API key from the Authorization header or `api_key`
query parameter. The middleware uses this to gate access to API routes when
a router_api_key is configured.
"""
auth_header = request.headers.get("Authorization")
if auth_header and auth_header.lower().startswith("bearer "):
return auth_header.split(" ", 1)[1].strip()
query_key = request.query_params.get("api_key")
if query_key:
return query_key
return None
def _strip_api_key_from_scope(request: Request) -> None:
"""
Remove api_key from the ASGI scope query string to avoid leaking it in logs.
"""
scope = request.scope
raw_qs = scope.get("query_string", b"")
if not raw_qs:
return
params = parse_qsl(raw_qs.decode("utf-8"), keep_blank_values=True)
filtered = [(k, v) for (k, v) in params if k != "api_key"]
scope["query_string"] = urlencode(filtered).encode("utf-8")
@app.middleware("http")
async def enforce_router_api_key(request: Request, call_next):
"""
Enforce the optional NOMYO Router API key for all non-static requests.
When `config.router_api_key` is set, clients must supply the key either in
the Authorization header (`Bearer <key>`) or as `api_key` query parameter.
"""
expected_key = config.router_api_key
if not expected_key or request.method == "OPTIONS":
return await call_next(request)
path = request.url.path
if path.startswith("/static") or path in {"/", "/favicon.ico"}:
return await call_next(request)
provided_key = _extract_router_api_key(request)
# Strip the api_key query param from scope so access logs do not leak it
_strip_api_key_from_scope(request)
if provided_key is None:
return JSONResponse(
content={"detail": "Missing NOMYO Router API key"},
status_code=401,
)
if not secrets.compare_digest(str(provided_key), str(expected_key)):
return JSONResponse(
content={"detail": "Invalid NOMYO Router API key"},
status_code=403,
)
return await call_next(request)
# -------------------------------------------------------------
# 3. Global state: perendpoint permodel active connection counters
# -------------------------------------------------------------
@ -165,7 +267,7 @@ def _is_fresh(cached_at: float, ttl: int) -> bool:
async def _ensure_success(resp: aiohttp.ClientResponse) -> None:
if resp.status >= 400:
text = await resp.text()
raise HTTPException(status_code=resp.status, detail=text)
raise HTTPException(status_code=resp.status, detail=_mask_secrets(text))
def _format_connection_issue(url: str, error: Exception) -> str:
"""
@ -1809,7 +1911,10 @@ async def config_proxy(request: Request):
return {"url": url, "status": "error", "detail": detail}
results = await asyncio.gather(*[check_endpoint(ep) for ep in config.endpoints])
return {"endpoints": results}
return {
"endpoints": results,
"require_router_api_key": bool(config.router_api_key),
}
# -------------------------------------------------------------
# 21. API route OpenAI compatible Embedding

View file

@ -11,6 +11,12 @@
color: #333;
padding: 20px;
}
body.auth-locked {
background: #bfbfbf;
}
body.auth-locked > *:not(#api-key-modal) {
display: none !important;
}
.dark-mode {
filter: invert(100%);
}
@ -167,6 +173,29 @@
cursor: pointer;
font-size: 1.5rem;
}
#api-key-modal .modal-content {
width: 420px;
height: auto;
max-width: 90%;
}
#api-key-modal .modal-message {
margin: 0.5rem 0;
}
#api-key-modal input[type="password"] {
width: 100%;
padding: 0.6rem;
box-sizing: border-box;
}
#api-key-modal .modal-actions {
display: flex;
justify-content: flex-end;
gap: 0.5rem;
margin-top: 0.5rem;
}
#api-key-indicator {
font-size: 0.9rem;
color: #555;
}
/* ---------- Usage Chart ---------- */
.usage-chart {
margin-top: 20px;
@ -269,7 +298,8 @@
/></a>
<div class="header-row">
<h1>Router Dashboard</h1>
<button id="total-tokens-btn">Stats Total</button><span id="aggregation-status" class="loading" style="margin-left:8px;"></span>
<button id="total-tokens-btn">Stats Total</button>
<span id="aggregation-status" class="loading" style="margin-left:8px;"></span>
</div>
<button onclick="toggleDarkMode()" id="dark-mode-button">
@ -353,14 +383,109 @@
</table>
</div>
<script>
<script>
let psRows = new Map();
let statsModal = null;
let statsChart = null;
let rawTimeSeries = null;
let totalTokensChart = null;
let usageSource = null;
// Global placeholders for stats modal handling
let statsModal = null; // stats modal element
let statsChart = null; // Chart.js instance inside the modal
let rawTimeSeries = null; // raw timeseries data for the current model
let totalTokensChart = null; // Chart.js instance for total tokens modal
const API_KEY_STORAGE_KEY = "nomyo-router-api-key";
let apiKeyWaiters = [];
function getStoredApiKey() {
return localStorage.getItem(API_KEY_STORAGE_KEY) || "";
}
function setStoredApiKey(key) {
if (key) {
localStorage.setItem(API_KEY_STORAGE_KEY, key);
} else {
localStorage.removeItem(API_KEY_STORAGE_KEY);
}
updateApiKeyIndicator();
}
function updateApiKeyIndicator() {
const indicator = document.getElementById("api-key-indicator");
if (!indicator) return;
const hasKey = !!getStoredApiKey();
indicator.textContent = hasKey ? "API key set" : "API key not set";
indicator.style.color = hasKey ? "green" : "#b22222";
}
function buildAuthedUrl(url) {
const key = getStoredApiKey();
if (!key) return url;
try {
const u = new URL(url, window.location.origin);
if (!u.searchParams.has("api_key")) {
u.searchParams.set("api_key", key);
}
return u.toString();
} catch (err) {
return url;
}
}
function showApiKeyModal(reasonText) {
const overlay = document.getElementById("api-key-modal");
if (!overlay) return Promise.resolve();
document.body.classList.add("auth-locked");
const reason = document.getElementById("api-key-reason");
if (reason) {
reason.textContent =
reasonText ||
"Enter the NOMYO Router API key to continue.";
}
const status = document.getElementById("api-key-status");
if (status) {
status.textContent = "";
}
const input = document.getElementById("api-key-input");
if (input) {
input.value = getStoredApiKey();
setTimeout(() => input.focus(), 10);
}
overlay.style.display = "flex";
return new Promise((resolve) => apiKeyWaiters.push(resolve));
}
function closeApiKeyModal(statusMessage) {
const overlay = document.getElementById("api-key-modal");
const status = document.getElementById("api-key-status");
if (status) {
status.textContent = statusMessage || "";
}
if (overlay) {
overlay.style.display = "none";
}
if (getStoredApiKey()) {
document.body.classList.remove("auth-locked");
}
while (apiKeyWaiters.length) {
const resolve = apiKeyWaiters.pop();
resolve();
}
}
async function authedFetch(url, options = {}, allowRetry = true) {
const headers = new Headers(options.headers || {});
const key = getStoredApiKey();
if (key) {
headers.set("Authorization", `Bearer ${key}`);
}
const response = await fetch(url, { ...options, headers });
if ((response.status === 401 || response.status === 403) && allowRetry) {
await showApiKeyModal("Enter the NOMYO Router API key to continue.");
return authedFetch(url, options, false);
}
return response;
}
/* Integrated modal initialization and close handling into the main load block */
@ -505,11 +630,11 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
chart.data.labels = labels;
chart.data.datasets[0].data = inputData;
chart.data.datasets[1].data = outputData;
chart.update();
chart.update();
}
/* ---------- Utility ---------- */
async function fetchJSON(url) {
const resp = await fetch(url);
async function fetchJSON(url, options = {}) {
const resp = await authedFetch(url, options);
if (!resp.ok) {
throw new Error(`Failed ${url}: ${resp.status}`);
}
@ -524,6 +649,9 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
async function loadEndpoints() {
try {
const data = await fetchJSON("/api/config");
if (data.require_router_api_key && !getStoredApiKey()) {
showApiKeyModal("Enter the NOMYO Router API key to load the dashboard.");
}
const body = document.getElementById("endpoints-body");
body.innerHTML = data.endpoints
.map((e) => {
@ -580,7 +708,7 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
);
if (!dest) return;
try {
const resp = await fetch(
const resp = await authedFetch(
`/api/copy?source=${encodeURIComponent(source)}&destination=${encodeURIComponent(dest)}`,
{ method: "POST" },
);
@ -613,7 +741,7 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
);
if (ok) {
try {
const resp = await fetch(
const resp = await authedFetch(
`/api/delete?model=${encodeURIComponent(model)}`,
{ method: "DELETE" },
);
@ -689,8 +817,12 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
return Math.abs(hash);
}
async function loadUsage() {
// Create the EventSource once and keep it around
const source = new EventSource("/api/usage-stream");
if (usageSource) {
usageSource.close();
usageSource = null;
}
const source = new EventSource(buildAuthedUrl("/api/usage-stream"));
usageSource = source;
// Helper that receives the payload and renders the chart
const renderChart = (data) => {
@ -734,7 +866,8 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
psRows.forEach((row, model) => {
let tokenTotal = 0;
for (const ep in tokens) {
tokenTotal += tokens[ep][model] || 0;
const endpointTokens = tokens[ep] || {};
tokenTotal += endpointTokens[model] || 0;
}
const tokenCell = row.querySelector(".token-usage");
if (tokenCell) tokenCell.textContent = tokenTotal;
@ -744,14 +877,44 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
}
};
source.onerror = (err) => {
source.onerror = async (err) => {
console.error("SSE connection error. Retrying...", err);
source.close();
await showApiKeyModal("Enter the NOMYO Router API key to view live usage.");
loadUsage();
};
window.addEventListener("beforeunload", () => source.close());
}
/* ---------- Init ---------- */
window.addEventListener("load", () => {
updateApiKeyIndicator();
const apiKeyModal = document.getElementById("api-key-modal");
if (apiKeyModal) {
apiKeyModal.addEventListener("click", (e) => {
if (e.target === apiKeyModal || e.target.matches(".close-btn")) {
closeApiKeyModal();
}
});
}
const saveKeyBtn = document.getElementById("api-key-save");
if (saveKeyBtn) {
saveKeyBtn.addEventListener("click", () => {
const key = document.getElementById("api-key-input")?.value.trim();
setStoredApiKey(key);
closeApiKeyModal(key ? "API key saved." : "API key cleared.");
loadUsage();
});
}
const clearKeyBtn = document.getElementById("api-key-clear");
if (clearKeyBtn) {
clearKeyBtn.addEventListener("click", () => {
setStoredApiKey("");
closeApiKeyModal("API key cleared.");
loadUsage();
});
}
loadEndpoints();
loadTags();
loadPS();
@ -765,7 +928,7 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
e.preventDefault();
const model = e.target.dataset.model;
try {
const resp = await fetch(
const resp = await authedFetch(
`/api/show?model=${encodeURIComponent(model)}`,
{ method: "POST" },
);
@ -802,7 +965,7 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
return;
}
try {
const resp = await fetch(
const resp = await authedFetch(
`/api/pull?model=${encodeURIComponent(model)}`,
{ method: "POST" },
);
@ -836,7 +999,7 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
e.preventDefault();
const model = e.target.dataset.model;
try {
const resp = await fetch(
const resp = await authedFetch(
`/api/stats?model=${encodeURIComponent(model)}`,
{ method: "POST" },
);
@ -1003,14 +1166,14 @@ document.addEventListener('DOMContentLoaded', () => {
if (totalBtn) {
totalBtn.addEventListener('click', async () => {
try {
const resp = await fetch('/api/token_counts');
const resp = await authedFetch('/api/token_counts');
const data = await resp.json();
const modal = document.getElementById('total-tokens-modal');
const numberEl = document.getElementById('total-tokens-number');
numberEl.textContent = data.total_tokens;
document.getElementById('aggregation-status').textContent = 'Aggregating...';
try {
const aggResp = await fetch('/api/aggregate_time_series_days', {
const aggResp = await authedFetch('/api/aggregate_time_series_days', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ days: 30 , trim_old: true})
@ -1119,6 +1282,25 @@ document.addEventListener('DOMContentLoaded', () => {
});
</script>
<div id="api-key-modal" class="modal">
<div class="modal-content">
<span class="close-btn">&times;</span>
<h2>Router API Key</h2>
<p id="api-key-reason" class="modal-message">Enter the NOMYO Router API key to continue.</p>
<input
id="api-key-input"
type="password"
placeholder="NOMYO Router API key"
autocomplete="off"
/>
<div class="modal-actions">
<button id="api-key-clear">Clear</button>
<button id="api-key-save">Save</button>
</div>
<p id="api-key-status" class="loading"></p>
</div>
</div>
<div id="show-modal" class="modal">
<div class="modal-content">
<span class="close-btn">&times;</span>