add: Optional router-level API key that gates router/API/web UI access
Optional router-level API key that gates router/API/web UI access (leave empty to disable) ## Supplying the router API key If you set `nomyo-router-api-key` in `config.yaml` (or `NOMYO_ROUTER_API_KEY` env), every request to NOMYO Router must include the key: - HTTP header (recommended): `Authorization: Bearer <router_key>` - Query param (fallback): `?api_key=<router_key>` Examples: ```bash curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags curl "http://localhost:12434/api/tags?api_key=$NOMYO_ROUTER_API_KEY" ```
This commit is contained in:
parent
6828411f95
commit
eca4a92a33
9 changed files with 412 additions and 25 deletions
19
README.md
19
README.md
|
|
@ -23,6 +23,9 @@ endpoints:
|
||||||
# Maximum concurrent connections *per endpoint‑model pair*
|
# Maximum concurrent connections *per endpoint‑model pair*
|
||||||
max_concurrent_connections: 2
|
max_concurrent_connections: 2
|
||||||
|
|
||||||
|
# Optional router-level API key to lock down router + dashboard (leave empty to disable)
|
||||||
|
nomyo-router-api-key: ""
|
||||||
|
|
||||||
# API keys for remote endpoints
|
# API keys for remote endpoints
|
||||||
# Set an environment variable like OPENAI_KEY
|
# Set an environment variable like OPENAI_KEY
|
||||||
# Confirm endpoints are exactly as in endpoints block
|
# Confirm endpoints are exactly as in endpoints block
|
||||||
|
|
@ -45,6 +48,8 @@ pip3 install -r requirements.txt
|
||||||
|
|
||||||
```
|
```
|
||||||
export OPENAI_KEY=YOUR_SECRET_API_KEY
|
export OPENAI_KEY=YOUR_SECRET_API_KEY
|
||||||
|
# Optional: router-level key (clients must send Authorization: Bearer)
|
||||||
|
# export NOMYO_ROUTER_API_KEY=YOUR_ROUTER_KEY
|
||||||
```
|
```
|
||||||
|
|
||||||
finally you can
|
finally you can
|
||||||
|
|
@ -91,3 +96,17 @@ This way the Ollama backend servers are utilized more efficient than by simply u
|
||||||

|

|
||||||
|
|
||||||
NOMYO Router also supports OpenAI API compatible v1 backend servers.
|
NOMYO Router also supports OpenAI API compatible v1 backend servers.
|
||||||
|
|
||||||
|
|
||||||
|
## Supplying the router API key
|
||||||
|
|
||||||
|
If you set `nomyo-router-api-key` in `config.yaml` (or `NOMYO_ROUTER_API_KEY` env), every request to NOMYO Router must include the key:
|
||||||
|
|
||||||
|
- HTTP header (recommended): `Authorization: Bearer <router_key>`
|
||||||
|
- Query param (fallback): `?api_key=<router_key>`
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
```bash
|
||||||
|
curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags
|
||||||
|
curl "http://localhost:12434/api/tags?api_key=$NOMYO_ROUTER_API_KEY"
|
||||||
|
```
|
||||||
|
|
|
||||||
|
|
@ -8,6 +8,9 @@ endpoints:
|
||||||
# Maximum concurrent connections *per endpoint‑model pair* (equals to OLLAMA_NUM_PARALLEL)
|
# Maximum concurrent connections *per endpoint‑model pair* (equals to OLLAMA_NUM_PARALLEL)
|
||||||
max_concurrent_connections: 2
|
max_concurrent_connections: 2
|
||||||
|
|
||||||
|
# Optional router-level API key that gates router/API/web UI access (leave empty to disable)
|
||||||
|
nomyo-router-api-key: ""
|
||||||
|
|
||||||
# API keys for remote endpoints
|
# API keys for remote endpoints
|
||||||
# Set an environment variable like OPENAI_KEY
|
# Set an environment variable like OPENAI_KEY
|
||||||
# Confirm endpoints are exactly as in endpoints block
|
# Confirm endpoints are exactly as in endpoints block
|
||||||
|
|
|
||||||
|
|
@ -36,6 +36,8 @@ doc/
|
||||||
endpoints:
|
endpoints:
|
||||||
- http://localhost:11434
|
- http://localhost:11434
|
||||||
max_concurrent_connections: 2
|
max_concurrent_connections: 2
|
||||||
|
# Optional router-level API key (leave blank to disable)
|
||||||
|
nomyo-router-api-key: ""
|
||||||
```
|
```
|
||||||
3. **Run the router**:
|
3. **Run the router**:
|
||||||
|
|
||||||
|
|
@ -135,3 +137,15 @@ For additional help:
|
||||||
5. **Scale as needed** by adding more endpoints
|
5. **Scale as needed** by adding more endpoints
|
||||||
|
|
||||||
Happy routing! 🚀
|
Happy routing! 🚀
|
||||||
|
|
||||||
|
|
||||||
|
## Router API key usage
|
||||||
|
|
||||||
|
If the router API key is set (`NOMYO_ROUTER_API_KEY` env or `nomyo-router-api-key` in config), include it in every request:
|
||||||
|
- Header (preferred): Authorization: Bearer <router_key>
|
||||||
|
- Query param: ?api_key=<router_key>
|
||||||
|
|
||||||
|
Example:
|
||||||
|
```bash
|
||||||
|
curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags
|
||||||
|
```
|
||||||
|
|
|
||||||
|
|
@ -14,6 +14,9 @@ endpoints:
|
||||||
|
|
||||||
# Maximum concurrent connections *per endpoint‑model pair*
|
# Maximum concurrent connections *per endpoint‑model pair*
|
||||||
max_concurrent_connections: 2
|
max_concurrent_connections: 2
|
||||||
|
|
||||||
|
# Optional router-level API key to secure the router and dashboard (leave blank to disable)
|
||||||
|
nomyo-router-api-key: ""
|
||||||
```
|
```
|
||||||
|
|
||||||
### Complete Example
|
### Complete Example
|
||||||
|
|
@ -29,6 +32,9 @@ endpoints:
|
||||||
# Maximum concurrent connections *per endpoint‑model pair* (equals to OLLAMA_NUM_PARALLEL)
|
# Maximum concurrent connections *per endpoint‑model pair* (equals to OLLAMA_NUM_PARALLEL)
|
||||||
max_concurrent_connections: 2
|
max_concurrent_connections: 2
|
||||||
|
|
||||||
|
# Optional router-level API key to secure the router and dashboard (leave blank to disable)
|
||||||
|
nomyo-router-api-key: ""
|
||||||
|
|
||||||
# API keys for remote endpoints
|
# API keys for remote endpoints
|
||||||
# Set an environment variable like OPENAI_KEY
|
# Set an environment variable like OPENAI_KEY
|
||||||
# Confirm endpoints are exactly as in endpoints block
|
# Confirm endpoints are exactly as in endpoints block
|
||||||
|
|
@ -80,6 +86,21 @@ max_concurrent_connections: 4
|
||||||
- When this limit is reached, the router will route requests to other endpoints with available capacity
|
- When this limit is reached, the router will route requests to other endpoints with available capacity
|
||||||
- Higher values allow more parallel requests but may increase memory usage
|
- Higher values allow more parallel requests but may increase memory usage
|
||||||
|
|
||||||
|
### `router_api_key`
|
||||||
|
|
||||||
|
**Type**: `str` (optional)
|
||||||
|
|
||||||
|
**Description**: Shared secret that gates access to the NOMYO Router APIs and dashboard. When set, clients must send `Authorization: Bearer <key>` or an `api_key` query parameter.
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
```yaml
|
||||||
|
nomyo-router-api-key: "super-secret-value"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Notes**:
|
||||||
|
- Leave this blank or omit it to disable router-level authentication.
|
||||||
|
- You can also set the `NOMYO_ROUTER_API_KEY` environment variable to avoid storing the key in plain text.
|
||||||
|
|
||||||
### `api_keys`
|
### `api_keys`
|
||||||
|
|
||||||
**Type**: `dict[str, str]`
|
**Type**: `dict[str, str]`
|
||||||
|
|
@ -118,6 +139,15 @@ export NOMYO_ROUTER_CONFIG_PATH=/etc/nomyo-router/config.yaml
|
||||||
export NOMYO_ROUTER_DB_PATH=/var/lib/nomyo-router/token_counts.db
|
export NOMYO_ROUTER_DB_PATH=/var/lib/nomyo-router/token_counts.db
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### `NOMYO_ROUTER_API_KEY`
|
||||||
|
|
||||||
|
**Description**: Router-level API key. When set, all router endpoints and the dashboard require this key via `Authorization: Bearer <key>` or the `api_key` query parameter.
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
```bash
|
||||||
|
export NOMYO_ROUTER_API_KEY=your_router_api_key
|
||||||
|
```
|
||||||
|
|
||||||
### API-Specific Keys
|
### API-Specific Keys
|
||||||
|
|
||||||
You can set API keys directly as environment variables:
|
You can set API keys directly as environment variables:
|
||||||
|
|
@ -195,3 +225,15 @@ The configuration is loaded at startup and cannot be changed without restarting
|
||||||
## Example Configurations
|
## Example Configurations
|
||||||
|
|
||||||
See the [examples](examples/) directory for ready-to-use configuration examples.
|
See the [examples](examples/) directory for ready-to-use configuration examples.
|
||||||
|
|
||||||
|
|
||||||
|
### Using the router API key
|
||||||
|
|
||||||
|
When `router_api_key`/`NOMYO_ROUTER_API_KEY` is set, clients must send it on every request:
|
||||||
|
- Header (recommended): Authorization: Bearer <router_key>
|
||||||
|
- Query param (fallback): ?api_key=<router_key>
|
||||||
|
|
||||||
|
Example:
|
||||||
|
```bash
|
||||||
|
curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags
|
||||||
|
```
|
||||||
|
|
|
||||||
|
|
@ -217,6 +217,7 @@ data:
|
||||||
endpoints:
|
endpoints:
|
||||||
- http://ollama-service:11434
|
- http://ollama-service:11434
|
||||||
max_concurrent_connections: 2
|
max_concurrent_connections: 2
|
||||||
|
nomyo-router-api-key: ""
|
||||||
---
|
---
|
||||||
apiVersion: v1
|
apiVersion: v1
|
||||||
kind: PersistentVolumeClaim
|
kind: PersistentVolumeClaim
|
||||||
|
|
@ -283,6 +284,9 @@ endpoints:
|
||||||
|
|
||||||
max_concurrent_connections: 4
|
max_concurrent_connections: 4
|
||||||
|
|
||||||
|
# Optional router-level API key to secure the router and dashboard (leave blank to disable)
|
||||||
|
nomyo-router-api-key: ""
|
||||||
|
|
||||||
api_keys:
|
api_keys:
|
||||||
"https://api.openai.com/v1": "${OPENAI_KEY}"
|
"https://api.openai.com/v1": "${OPENAI_KEY}"
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
@ -6,6 +6,9 @@ endpoints:
|
||||||
|
|
||||||
max_concurrent_connections: 2
|
max_concurrent_connections: 2
|
||||||
|
|
||||||
|
# Optional router-level API key to secure the router and dashboard (leave blank to disable)
|
||||||
|
nomyo-router-api-key: ""
|
||||||
|
|
||||||
# Multi-endpoint configuration with local Ollama instances
|
# Multi-endpoint configuration with local Ollama instances
|
||||||
# endpoints:
|
# endpoints:
|
||||||
# - http://ollama-worker1:11434
|
# - http://ollama-worker1:11434
|
||||||
|
|
|
||||||
15
doc/usage.md
15
doc/usage.md
|
|
@ -21,6 +21,9 @@ endpoints:
|
||||||
- http://localhost:11434
|
- http://localhost:11434
|
||||||
|
|
||||||
max_concurrent_connections: 2
|
max_concurrent_connections: 2
|
||||||
|
|
||||||
|
# Optional router-level API key (leave blank to disable)
|
||||||
|
nomyo-router-api-key: ""
|
||||||
```
|
```
|
||||||
|
|
||||||
### 3. Run the Router
|
### 3. Run the Router
|
||||||
|
|
@ -346,3 +349,15 @@ print(f"Available models: {[m.id for m in response.data]}")
|
||||||
## Examples
|
## Examples
|
||||||
|
|
||||||
See the [examples](examples/) directory for complete integration examples.
|
See the [examples](examples/) directory for complete integration examples.
|
||||||
|
|
||||||
|
|
||||||
|
### Authentication to NOMYO Router
|
||||||
|
|
||||||
|
If a router API key is configured, include it with each request:
|
||||||
|
- Header: Authorization: Bearer <router_key>
|
||||||
|
- Query: ?api_key=<router_key>
|
||||||
|
|
||||||
|
Example (tags):
|
||||||
|
```bash
|
||||||
|
curl -H "Authorization: Bearer $NOMYO_ROUTER_API_KEY" http://localhost:12434/api/tags
|
||||||
|
```
|
||||||
|
|
|
||||||
113
router.py
113
router.py
|
|
@ -6,14 +6,14 @@ version: 0.5
|
||||||
license: AGPL
|
license: AGPL
|
||||||
"""
|
"""
|
||||||
# -------------------------------------------------------------
|
# -------------------------------------------------------------
|
||||||
import orjson, time, asyncio, yaml, ollama, openai, os, re, aiohttp, ssl, random, base64, io, enhance
|
import orjson, time, asyncio, yaml, ollama, openai, os, re, aiohttp, ssl, random, base64, io, enhance, secrets
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
# Directory containing static files (relative to this script)
|
# Directory containing static files (relative to this script)
|
||||||
STATIC_DIR = Path(__file__).parent / "static"
|
STATIC_DIR = Path(__file__).parent / "static"
|
||||||
from typing import Dict, Set, List, Optional
|
from typing import Dict, Set, List, Optional
|
||||||
from urllib.parse import urlparse
|
from urllib.parse import urlparse, parse_qsl, urlencode
|
||||||
from fastapi import FastAPI, Request, HTTPException
|
from fastapi import FastAPI, Request, HTTPException
|
||||||
from fastapi_sse import sse_handler
|
from fastapi_sse import sse_handler
|
||||||
from fastapi.staticfiles import StaticFiles
|
from fastapi.staticfiles import StaticFiles
|
||||||
|
|
@ -41,6 +41,20 @@ _subscribers: Set[asyncio.Queue] = set()
|
||||||
_subscribers_lock = asyncio.Lock()
|
_subscribers_lock = asyncio.Lock()
|
||||||
token_queue: asyncio.Queue[tuple[str, str, int, int]] = asyncio.Queue()
|
token_queue: asyncio.Queue[tuple[str, str, int, int]] = asyncio.Queue()
|
||||||
|
|
||||||
|
# -------------------------------------------------------------
|
||||||
|
# Secret handling
|
||||||
|
# -------------------------------------------------------------
|
||||||
|
def _mask_secrets(text: str) -> str:
|
||||||
|
"""
|
||||||
|
Mask common API key patterns to avoid leaking secrets in logs or error payloads.
|
||||||
|
"""
|
||||||
|
if not text:
|
||||||
|
return text
|
||||||
|
# OpenAI-style keys (sk-...) and generic "api key" mentions
|
||||||
|
text = re.sub(r"sk-[A-Za-z0-9]{4}[A-Za-z0-9_-]*", "sk-***redacted***", text)
|
||||||
|
text = re.sub(r"(?i)(api[-_ ]key\\s*[:=]\\s*)([^\\s]+)", r"\\1***redacted***", text)
|
||||||
|
return text
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Globals
|
# Globals
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
@ -78,6 +92,8 @@ class Config(BaseSettings):
|
||||||
max_concurrent_connections: int = 1
|
max_concurrent_connections: int = 1
|
||||||
|
|
||||||
api_keys: Dict[str, str] = Field(default_factory=dict)
|
api_keys: Dict[str, str] = Field(default_factory=dict)
|
||||||
|
# Optional router-level API key used to gate access to this service and dashboard
|
||||||
|
router_api_key: Optional[str] = Field(default=None, env="NOMYO_ROUTER_API_KEY")
|
||||||
|
|
||||||
# Database configuration
|
# Database configuration
|
||||||
db_path: str = Field(default=os.getenv("NOMYO_ROUTER_DB_PATH", "token_counts.db"))
|
db_path: str = Field(default=os.getenv("NOMYO_ROUTER_DB_PATH", "token_counts.db"))
|
||||||
|
|
@ -108,6 +124,29 @@ class Config(BaseSettings):
|
||||||
with path.open("r", encoding="utf-8") as fp:
|
with path.open("r", encoding="utf-8") as fp:
|
||||||
data = yaml.safe_load(fp) or {}
|
data = yaml.safe_load(fp) or {}
|
||||||
cleaned = cls._expand_env_refs(data)
|
cleaned = cls._expand_env_refs(data)
|
||||||
|
if isinstance(cleaned, dict):
|
||||||
|
# Accept hyphenated config key and map it to the field name
|
||||||
|
key_aliases = [
|
||||||
|
# canonical field name
|
||||||
|
"router_api_key",
|
||||||
|
# lowercase, hyphen/underscore variants
|
||||||
|
"nomyo-router-api-key",
|
||||||
|
"nomyo_router_api_key",
|
||||||
|
"nomyo-router_api_key",
|
||||||
|
"nomyo_router-api_key",
|
||||||
|
# uppercase env-style variants
|
||||||
|
"NOMYO-ROUTER_API_KEY",
|
||||||
|
"NOMYO_ROUTER_API_KEY",
|
||||||
|
]
|
||||||
|
for alias in key_aliases:
|
||||||
|
if alias in cleaned:
|
||||||
|
cleaned["router_api_key"] = cleaned.get("router_api_key", cleaned.pop(alias))
|
||||||
|
break
|
||||||
|
# If not present in YAML (or empty), fall back to env var explicitly
|
||||||
|
if not cleaned.get("router_api_key"):
|
||||||
|
env_key = os.getenv("NOMYO_ROUTER_API_KEY")
|
||||||
|
if env_key:
|
||||||
|
cleaned["router_api_key"] = env_key
|
||||||
return cls(**cleaned)
|
return cls(**cleaned)
|
||||||
return cls()
|
return cls()
|
||||||
|
|
||||||
|
|
@ -145,6 +184,69 @@ default_headers={
|
||||||
"X-Title": "NOMYO Router",
|
"X-Title": "NOMYO Router",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# -------------------------------------------------------------
|
||||||
|
# Router-level authentication (optional)
|
||||||
|
# -------------------------------------------------------------
|
||||||
|
def _extract_router_api_key(request: Request) -> Optional[str]:
|
||||||
|
"""
|
||||||
|
Extract the provided router API key from the Authorization header or `api_key`
|
||||||
|
query parameter. The middleware uses this to gate access to API routes when
|
||||||
|
a router_api_key is configured.
|
||||||
|
"""
|
||||||
|
auth_header = request.headers.get("Authorization")
|
||||||
|
if auth_header and auth_header.lower().startswith("bearer "):
|
||||||
|
return auth_header.split(" ", 1)[1].strip()
|
||||||
|
query_key = request.query_params.get("api_key")
|
||||||
|
if query_key:
|
||||||
|
return query_key
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_api_key_from_scope(request: Request) -> None:
|
||||||
|
"""
|
||||||
|
Remove api_key from the ASGI scope query string to avoid leaking it in logs.
|
||||||
|
"""
|
||||||
|
scope = request.scope
|
||||||
|
raw_qs = scope.get("query_string", b"")
|
||||||
|
if not raw_qs:
|
||||||
|
return
|
||||||
|
params = parse_qsl(raw_qs.decode("utf-8"), keep_blank_values=True)
|
||||||
|
filtered = [(k, v) for (k, v) in params if k != "api_key"]
|
||||||
|
scope["query_string"] = urlencode(filtered).encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
@app.middleware("http")
|
||||||
|
async def enforce_router_api_key(request: Request, call_next):
|
||||||
|
"""
|
||||||
|
Enforce the optional NOMYO Router API key for all non-static requests.
|
||||||
|
When `config.router_api_key` is set, clients must supply the key either in
|
||||||
|
the Authorization header (`Bearer <key>`) or as `api_key` query parameter.
|
||||||
|
"""
|
||||||
|
expected_key = config.router_api_key
|
||||||
|
if not expected_key or request.method == "OPTIONS":
|
||||||
|
return await call_next(request)
|
||||||
|
|
||||||
|
path = request.url.path
|
||||||
|
if path.startswith("/static") or path in {"/", "/favicon.ico"}:
|
||||||
|
return await call_next(request)
|
||||||
|
|
||||||
|
provided_key = _extract_router_api_key(request)
|
||||||
|
# Strip the api_key query param from scope so access logs do not leak it
|
||||||
|
_strip_api_key_from_scope(request)
|
||||||
|
if provided_key is None:
|
||||||
|
return JSONResponse(
|
||||||
|
content={"detail": "Missing NOMYO Router API key"},
|
||||||
|
status_code=401,
|
||||||
|
)
|
||||||
|
|
||||||
|
if not secrets.compare_digest(str(provided_key), str(expected_key)):
|
||||||
|
return JSONResponse(
|
||||||
|
content={"detail": "Invalid NOMYO Router API key"},
|
||||||
|
status_code=403,
|
||||||
|
)
|
||||||
|
|
||||||
|
return await call_next(request)
|
||||||
|
|
||||||
# -------------------------------------------------------------
|
# -------------------------------------------------------------
|
||||||
# 3. Global state: per‑endpoint per‑model active connection counters
|
# 3. Global state: per‑endpoint per‑model active connection counters
|
||||||
# -------------------------------------------------------------
|
# -------------------------------------------------------------
|
||||||
|
|
@ -165,7 +267,7 @@ def _is_fresh(cached_at: float, ttl: int) -> bool:
|
||||||
async def _ensure_success(resp: aiohttp.ClientResponse) -> None:
|
async def _ensure_success(resp: aiohttp.ClientResponse) -> None:
|
||||||
if resp.status >= 400:
|
if resp.status >= 400:
|
||||||
text = await resp.text()
|
text = await resp.text()
|
||||||
raise HTTPException(status_code=resp.status, detail=text)
|
raise HTTPException(status_code=resp.status, detail=_mask_secrets(text))
|
||||||
|
|
||||||
def _format_connection_issue(url: str, error: Exception) -> str:
|
def _format_connection_issue(url: str, error: Exception) -> str:
|
||||||
"""
|
"""
|
||||||
|
|
@ -1809,7 +1911,10 @@ async def config_proxy(request: Request):
|
||||||
return {"url": url, "status": "error", "detail": detail}
|
return {"url": url, "status": "error", "detail": detail}
|
||||||
|
|
||||||
results = await asyncio.gather(*[check_endpoint(ep) for ep in config.endpoints])
|
results = await asyncio.gather(*[check_endpoint(ep) for ep in config.endpoints])
|
||||||
return {"endpoints": results}
|
return {
|
||||||
|
"endpoints": results,
|
||||||
|
"require_router_api_key": bool(config.router_api_key),
|
||||||
|
}
|
||||||
|
|
||||||
# -------------------------------------------------------------
|
# -------------------------------------------------------------
|
||||||
# 21. API route – OpenAI compatible Embedding
|
# 21. API route – OpenAI compatible Embedding
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,12 @@
|
||||||
color: #333;
|
color: #333;
|
||||||
padding: 20px;
|
padding: 20px;
|
||||||
}
|
}
|
||||||
|
body.auth-locked {
|
||||||
|
background: #bfbfbf;
|
||||||
|
}
|
||||||
|
body.auth-locked > *:not(#api-key-modal) {
|
||||||
|
display: none !important;
|
||||||
|
}
|
||||||
.dark-mode {
|
.dark-mode {
|
||||||
filter: invert(100%);
|
filter: invert(100%);
|
||||||
}
|
}
|
||||||
|
|
@ -167,6 +173,29 @@
|
||||||
cursor: pointer;
|
cursor: pointer;
|
||||||
font-size: 1.5rem;
|
font-size: 1.5rem;
|
||||||
}
|
}
|
||||||
|
#api-key-modal .modal-content {
|
||||||
|
width: 420px;
|
||||||
|
height: auto;
|
||||||
|
max-width: 90%;
|
||||||
|
}
|
||||||
|
#api-key-modal .modal-message {
|
||||||
|
margin: 0.5rem 0;
|
||||||
|
}
|
||||||
|
#api-key-modal input[type="password"] {
|
||||||
|
width: 100%;
|
||||||
|
padding: 0.6rem;
|
||||||
|
box-sizing: border-box;
|
||||||
|
}
|
||||||
|
#api-key-modal .modal-actions {
|
||||||
|
display: flex;
|
||||||
|
justify-content: flex-end;
|
||||||
|
gap: 0.5rem;
|
||||||
|
margin-top: 0.5rem;
|
||||||
|
}
|
||||||
|
#api-key-indicator {
|
||||||
|
font-size: 0.9rem;
|
||||||
|
color: #555;
|
||||||
|
}
|
||||||
/* ---------- Usage Chart ---------- */
|
/* ---------- Usage Chart ---------- */
|
||||||
.usage-chart {
|
.usage-chart {
|
||||||
margin-top: 20px;
|
margin-top: 20px;
|
||||||
|
|
@ -269,7 +298,8 @@
|
||||||
/></a>
|
/></a>
|
||||||
<div class="header-row">
|
<div class="header-row">
|
||||||
<h1>Router Dashboard</h1>
|
<h1>Router Dashboard</h1>
|
||||||
<button id="total-tokens-btn">Stats Total</button><span id="aggregation-status" class="loading" style="margin-left:8px;"></span>
|
<button id="total-tokens-btn">Stats Total</button>
|
||||||
|
<span id="aggregation-status" class="loading" style="margin-left:8px;"></span>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<button onclick="toggleDarkMode()" id="dark-mode-button">
|
<button onclick="toggleDarkMode()" id="dark-mode-button">
|
||||||
|
|
@ -353,14 +383,109 @@
|
||||||
</table>
|
</table>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<script>
|
<script>
|
||||||
let psRows = new Map();
|
let psRows = new Map();
|
||||||
|
let statsModal = null;
|
||||||
|
let statsChart = null;
|
||||||
|
let rawTimeSeries = null;
|
||||||
|
let totalTokensChart = null;
|
||||||
|
let usageSource = null;
|
||||||
|
|
||||||
// Global placeholders for stats modal handling
|
const API_KEY_STORAGE_KEY = "nomyo-router-api-key";
|
||||||
let statsModal = null; // stats modal element
|
let apiKeyWaiters = [];
|
||||||
let statsChart = null; // Chart.js instance inside the modal
|
|
||||||
let rawTimeSeries = null; // raw time‑series data for the current model
|
function getStoredApiKey() {
|
||||||
let totalTokensChart = null; // Chart.js instance for total tokens modal
|
return localStorage.getItem(API_KEY_STORAGE_KEY) || "";
|
||||||
|
}
|
||||||
|
|
||||||
|
function setStoredApiKey(key) {
|
||||||
|
if (key) {
|
||||||
|
localStorage.setItem(API_KEY_STORAGE_KEY, key);
|
||||||
|
} else {
|
||||||
|
localStorage.removeItem(API_KEY_STORAGE_KEY);
|
||||||
|
}
|
||||||
|
updateApiKeyIndicator();
|
||||||
|
}
|
||||||
|
|
||||||
|
function updateApiKeyIndicator() {
|
||||||
|
const indicator = document.getElementById("api-key-indicator");
|
||||||
|
if (!indicator) return;
|
||||||
|
const hasKey = !!getStoredApiKey();
|
||||||
|
indicator.textContent = hasKey ? "API key set" : "API key not set";
|
||||||
|
indicator.style.color = hasKey ? "green" : "#b22222";
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildAuthedUrl(url) {
|
||||||
|
const key = getStoredApiKey();
|
||||||
|
if (!key) return url;
|
||||||
|
try {
|
||||||
|
const u = new URL(url, window.location.origin);
|
||||||
|
if (!u.searchParams.has("api_key")) {
|
||||||
|
u.searchParams.set("api_key", key);
|
||||||
|
}
|
||||||
|
return u.toString();
|
||||||
|
} catch (err) {
|
||||||
|
return url;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function showApiKeyModal(reasonText) {
|
||||||
|
const overlay = document.getElementById("api-key-modal");
|
||||||
|
if (!overlay) return Promise.resolve();
|
||||||
|
document.body.classList.add("auth-locked");
|
||||||
|
|
||||||
|
const reason = document.getElementById("api-key-reason");
|
||||||
|
if (reason) {
|
||||||
|
reason.textContent =
|
||||||
|
reasonText ||
|
||||||
|
"Enter the NOMYO Router API key to continue.";
|
||||||
|
}
|
||||||
|
const status = document.getElementById("api-key-status");
|
||||||
|
if (status) {
|
||||||
|
status.textContent = "";
|
||||||
|
}
|
||||||
|
|
||||||
|
const input = document.getElementById("api-key-input");
|
||||||
|
if (input) {
|
||||||
|
input.value = getStoredApiKey();
|
||||||
|
setTimeout(() => input.focus(), 10);
|
||||||
|
}
|
||||||
|
|
||||||
|
overlay.style.display = "flex";
|
||||||
|
return new Promise((resolve) => apiKeyWaiters.push(resolve));
|
||||||
|
}
|
||||||
|
|
||||||
|
function closeApiKeyModal(statusMessage) {
|
||||||
|
const overlay = document.getElementById("api-key-modal");
|
||||||
|
const status = document.getElementById("api-key-status");
|
||||||
|
if (status) {
|
||||||
|
status.textContent = statusMessage || "";
|
||||||
|
}
|
||||||
|
if (overlay) {
|
||||||
|
overlay.style.display = "none";
|
||||||
|
}
|
||||||
|
if (getStoredApiKey()) {
|
||||||
|
document.body.classList.remove("auth-locked");
|
||||||
|
}
|
||||||
|
while (apiKeyWaiters.length) {
|
||||||
|
const resolve = apiKeyWaiters.pop();
|
||||||
|
resolve();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function authedFetch(url, options = {}, allowRetry = true) {
|
||||||
|
const headers = new Headers(options.headers || {});
|
||||||
|
const key = getStoredApiKey();
|
||||||
|
if (key) {
|
||||||
|
headers.set("Authorization", `Bearer ${key}`);
|
||||||
|
}
|
||||||
|
const response = await fetch(url, { ...options, headers });
|
||||||
|
if ((response.status === 401 || response.status === 403) && allowRetry) {
|
||||||
|
await showApiKeyModal("Enter the NOMYO Router API key to continue.");
|
||||||
|
return authedFetch(url, options, false);
|
||||||
|
}
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
|
||||||
/* Integrated modal initialization and close handling into the main load block */
|
/* Integrated modal initialization and close handling into the main load block */
|
||||||
|
|
||||||
|
|
@ -505,11 +630,11 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
|
||||||
chart.data.labels = labels;
|
chart.data.labels = labels;
|
||||||
chart.data.datasets[0].data = inputData;
|
chart.data.datasets[0].data = inputData;
|
||||||
chart.data.datasets[1].data = outputData;
|
chart.data.datasets[1].data = outputData;
|
||||||
chart.update();
|
chart.update();
|
||||||
}
|
}
|
||||||
/* ---------- Utility ---------- */
|
/* ---------- Utility ---------- */
|
||||||
async function fetchJSON(url) {
|
async function fetchJSON(url, options = {}) {
|
||||||
const resp = await fetch(url);
|
const resp = await authedFetch(url, options);
|
||||||
if (!resp.ok) {
|
if (!resp.ok) {
|
||||||
throw new Error(`Failed ${url}: ${resp.status}`);
|
throw new Error(`Failed ${url}: ${resp.status}`);
|
||||||
}
|
}
|
||||||
|
|
@ -524,6 +649,9 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
|
||||||
async function loadEndpoints() {
|
async function loadEndpoints() {
|
||||||
try {
|
try {
|
||||||
const data = await fetchJSON("/api/config");
|
const data = await fetchJSON("/api/config");
|
||||||
|
if (data.require_router_api_key && !getStoredApiKey()) {
|
||||||
|
showApiKeyModal("Enter the NOMYO Router API key to load the dashboard.");
|
||||||
|
}
|
||||||
const body = document.getElementById("endpoints-body");
|
const body = document.getElementById("endpoints-body");
|
||||||
body.innerHTML = data.endpoints
|
body.innerHTML = data.endpoints
|
||||||
.map((e) => {
|
.map((e) => {
|
||||||
|
|
@ -580,7 +708,7 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
|
||||||
);
|
);
|
||||||
if (!dest) return;
|
if (!dest) return;
|
||||||
try {
|
try {
|
||||||
const resp = await fetch(
|
const resp = await authedFetch(
|
||||||
`/api/copy?source=${encodeURIComponent(source)}&destination=${encodeURIComponent(dest)}`,
|
`/api/copy?source=${encodeURIComponent(source)}&destination=${encodeURIComponent(dest)}`,
|
||||||
{ method: "POST" },
|
{ method: "POST" },
|
||||||
);
|
);
|
||||||
|
|
@ -613,7 +741,7 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
|
||||||
);
|
);
|
||||||
if (ok) {
|
if (ok) {
|
||||||
try {
|
try {
|
||||||
const resp = await fetch(
|
const resp = await authedFetch(
|
||||||
`/api/delete?model=${encodeURIComponent(model)}`,
|
`/api/delete?model=${encodeURIComponent(model)}`,
|
||||||
{ method: "DELETE" },
|
{ method: "DELETE" },
|
||||||
);
|
);
|
||||||
|
|
@ -689,8 +817,12 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
|
||||||
return Math.abs(hash);
|
return Math.abs(hash);
|
||||||
}
|
}
|
||||||
async function loadUsage() {
|
async function loadUsage() {
|
||||||
// Create the EventSource once and keep it around
|
if (usageSource) {
|
||||||
const source = new EventSource("/api/usage-stream");
|
usageSource.close();
|
||||||
|
usageSource = null;
|
||||||
|
}
|
||||||
|
const source = new EventSource(buildAuthedUrl("/api/usage-stream"));
|
||||||
|
usageSource = source;
|
||||||
|
|
||||||
// Helper that receives the payload and renders the chart
|
// Helper that receives the payload and renders the chart
|
||||||
const renderChart = (data) => {
|
const renderChart = (data) => {
|
||||||
|
|
@ -734,7 +866,8 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
|
||||||
psRows.forEach((row, model) => {
|
psRows.forEach((row, model) => {
|
||||||
let tokenTotal = 0;
|
let tokenTotal = 0;
|
||||||
for (const ep in tokens) {
|
for (const ep in tokens) {
|
||||||
tokenTotal += tokens[ep][model] || 0;
|
const endpointTokens = tokens[ep] || {};
|
||||||
|
tokenTotal += endpointTokens[model] || 0;
|
||||||
}
|
}
|
||||||
const tokenCell = row.querySelector(".token-usage");
|
const tokenCell = row.querySelector(".token-usage");
|
||||||
if (tokenCell) tokenCell.textContent = tokenTotal;
|
if (tokenCell) tokenCell.textContent = tokenTotal;
|
||||||
|
|
@ -744,14 +877,44 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
source.onerror = (err) => {
|
source.onerror = async (err) => {
|
||||||
console.error("SSE connection error. Retrying...", err);
|
console.error("SSE connection error. Retrying...", err);
|
||||||
|
source.close();
|
||||||
|
await showApiKeyModal("Enter the NOMYO Router API key to view live usage.");
|
||||||
|
loadUsage();
|
||||||
};
|
};
|
||||||
window.addEventListener("beforeunload", () => source.close());
|
window.addEventListener("beforeunload", () => source.close());
|
||||||
}
|
}
|
||||||
|
|
||||||
/* ---------- Init ---------- */
|
/* ---------- Init ---------- */
|
||||||
window.addEventListener("load", () => {
|
window.addEventListener("load", () => {
|
||||||
|
updateApiKeyIndicator();
|
||||||
|
const apiKeyModal = document.getElementById("api-key-modal");
|
||||||
|
if (apiKeyModal) {
|
||||||
|
apiKeyModal.addEventListener("click", (e) => {
|
||||||
|
if (e.target === apiKeyModal || e.target.matches(".close-btn")) {
|
||||||
|
closeApiKeyModal();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
const saveKeyBtn = document.getElementById("api-key-save");
|
||||||
|
if (saveKeyBtn) {
|
||||||
|
saveKeyBtn.addEventListener("click", () => {
|
||||||
|
const key = document.getElementById("api-key-input")?.value.trim();
|
||||||
|
setStoredApiKey(key);
|
||||||
|
closeApiKeyModal(key ? "API key saved." : "API key cleared.");
|
||||||
|
loadUsage();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
const clearKeyBtn = document.getElementById("api-key-clear");
|
||||||
|
if (clearKeyBtn) {
|
||||||
|
clearKeyBtn.addEventListener("click", () => {
|
||||||
|
setStoredApiKey("");
|
||||||
|
closeApiKeyModal("API key cleared.");
|
||||||
|
loadUsage();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
loadEndpoints();
|
loadEndpoints();
|
||||||
loadTags();
|
loadTags();
|
||||||
loadPS();
|
loadPS();
|
||||||
|
|
@ -765,7 +928,7 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
|
||||||
e.preventDefault();
|
e.preventDefault();
|
||||||
const model = e.target.dataset.model;
|
const model = e.target.dataset.model;
|
||||||
try {
|
try {
|
||||||
const resp = await fetch(
|
const resp = await authedFetch(
|
||||||
`/api/show?model=${encodeURIComponent(model)}`,
|
`/api/show?model=${encodeURIComponent(model)}`,
|
||||||
{ method: "POST" },
|
{ method: "POST" },
|
||||||
);
|
);
|
||||||
|
|
@ -802,7 +965,7 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
const resp = await fetch(
|
const resp = await authedFetch(
|
||||||
`/api/pull?model=${encodeURIComponent(model)}`,
|
`/api/pull?model=${encodeURIComponent(model)}`,
|
||||||
{ method: "POST" },
|
{ method: "POST" },
|
||||||
);
|
);
|
||||||
|
|
@ -836,7 +999,7 @@ function renderTimeSeriesChart(timeSeriesData, chart, minutes) {
|
||||||
e.preventDefault();
|
e.preventDefault();
|
||||||
const model = e.target.dataset.model;
|
const model = e.target.dataset.model;
|
||||||
try {
|
try {
|
||||||
const resp = await fetch(
|
const resp = await authedFetch(
|
||||||
`/api/stats?model=${encodeURIComponent(model)}`,
|
`/api/stats?model=${encodeURIComponent(model)}`,
|
||||||
{ method: "POST" },
|
{ method: "POST" },
|
||||||
);
|
);
|
||||||
|
|
@ -1003,14 +1166,14 @@ document.addEventListener('DOMContentLoaded', () => {
|
||||||
if (totalBtn) {
|
if (totalBtn) {
|
||||||
totalBtn.addEventListener('click', async () => {
|
totalBtn.addEventListener('click', async () => {
|
||||||
try {
|
try {
|
||||||
const resp = await fetch('/api/token_counts');
|
const resp = await authedFetch('/api/token_counts');
|
||||||
const data = await resp.json();
|
const data = await resp.json();
|
||||||
const modal = document.getElementById('total-tokens-modal');
|
const modal = document.getElementById('total-tokens-modal');
|
||||||
const numberEl = document.getElementById('total-tokens-number');
|
const numberEl = document.getElementById('total-tokens-number');
|
||||||
numberEl.textContent = data.total_tokens;
|
numberEl.textContent = data.total_tokens;
|
||||||
document.getElementById('aggregation-status').textContent = 'Aggregating...';
|
document.getElementById('aggregation-status').textContent = 'Aggregating...';
|
||||||
try {
|
try {
|
||||||
const aggResp = await fetch('/api/aggregate_time_series_days', {
|
const aggResp = await authedFetch('/api/aggregate_time_series_days', {
|
||||||
method: 'POST',
|
method: 'POST',
|
||||||
headers: { 'Content-Type': 'application/json' },
|
headers: { 'Content-Type': 'application/json' },
|
||||||
body: JSON.stringify({ days: 30 , trim_old: true})
|
body: JSON.stringify({ days: 30 , trim_old: true})
|
||||||
|
|
@ -1119,6 +1282,25 @@ document.addEventListener('DOMContentLoaded', () => {
|
||||||
});
|
});
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
|
<div id="api-key-modal" class="modal">
|
||||||
|
<div class="modal-content">
|
||||||
|
<span class="close-btn">×</span>
|
||||||
|
<h2>Router API Key</h2>
|
||||||
|
<p id="api-key-reason" class="modal-message">Enter the NOMYO Router API key to continue.</p>
|
||||||
|
<input
|
||||||
|
id="api-key-input"
|
||||||
|
type="password"
|
||||||
|
placeholder="NOMYO Router API key"
|
||||||
|
autocomplete="off"
|
||||||
|
/>
|
||||||
|
<div class="modal-actions">
|
||||||
|
<button id="api-key-clear">Clear</button>
|
||||||
|
<button id="api-key-save">Save</button>
|
||||||
|
</div>
|
||||||
|
<p id="api-key-status" class="loading"></p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
<div id="show-modal" class="modal">
|
<div id="show-modal" class="modal">
|
||||||
<div class="modal-content">
|
<div class="modal-content">
|
||||||
<span class="close-btn">×</span>
|
<span class="close-btn">×</span>
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue