alpha-nerd 2d9bbd06b5 doc/compliance.md aktualisiert

2026-06-10 13:25:17 +02:00

13 KiB

Raw Permalink Blame History

Compliance Plugin (aissurance.eu)

The compliance plugin turns the NOMYO Router from a pure traffic proxy into an AI compliance agent. When enabled, it periodically snapshots your AI-infrastructure landscape — model inventory, endpoint registry, and aggregate telemetry — and sends signed, structured evidence to aissurance.eu, the EU AI Act compliance platform.

The plugin runs in-process inside the Router; it is not a separate service. It only reads infrastructure metadata the Router already maintains for routing. No prompt or completion content is ever read, stored, or transmitted.

Non-critical by design. Any failure inside the plugin is logged and contained — it never affects request routing. If the upstream is unreachable, the Router keeps proxying and buffers evidence for later.

How it works

 Router state (models, endpoints, usage, cache, token DB)
        │  read-only snapshot, every polling_interval
        ▼
 Snapshot Collector → Payload Builder → HMAC-SHA256 Signer
        │
        ▼
 HTTPS POST (outbound only) ──────────────► aissurance.eu
        │  202 Accepted → done                /api/v1/discovery/receive
        │  network error / 5xx → encrypt + buffer to disk, retry next cycle
        ▼
 Encrypted offline buffer (replayed oldest-first on reconnect)

All communication is outbound from the Router. aissurance.eu never initiates contact, so the Router can run behind NAT, firewalls, or strict egress policies.

What is and isn't collected

This is the most important part to understand before enabling the plugin.

Collected (infrastructure metadata only)

Model inventory — name, version, quantization, weight size, serving endpoint
Endpoint registry — URL, type, health status, concurrency, TLS, auth method
Aggregate telemetry — active connections, uptime, global cache hit rate, 24h token totals

Never collected

Individual request payloads (prompt / completion content)
User identity or auth tokens from proxied requests
Conversation content passing through the proxy
End-user client IP addresses

Emitted as `null` (not yet instrumented)

The Router does not currently measure the following, so the plugin emits null rather than fabricating values. They are candidates for a future instrumentation phase:

Field	Where
`request_count_24h`, `error_count_24h`	per model
`avg_latency_ms`, `p99_latency_ms`	per model
`cache_hit_rate`, `avg_input_tokens`, `avg_output_tokens`	per model
`total_requests_24h`, `error_rate`	telemetry

Note on byte fields. The Router tracks tokens, not raw bytes. total_bytes_in_24h / total_bytes_out_24h therefore carry 24h token counts as the closest available signal.

Quick start

Obtain your AISSURANCE_KEY (and optionally AISSURANCE_TENANT_ID) from aissurance.eu during tenant signup. This is distinct from the Router's own nomyo-router-api-key — see Dual-key model.

Export the secret (never commit it):

export AISSURANCE_KEY="aiss_live_…"
export AISSURANCE_TENANT_ID="550e8400-e29b-41d4-a716-446655440000"   # optional

Add the compliance block to config.yaml:

compliance:
  enabled: true
  server_url: "https://www.aissurance.eu/api/v1/discovery/receive"
  api_key: "${AISSURANCE_KEY}"
  tenant_id: "${AISSURANCE_TENANT_ID}"
  polling_interval: 300

Restart the Router. On startup you should see structured logs:

{"event":"plugin_started","router_id":"router-prod-001","polling_interval":300,"buffered":0}

Configuration options

All options live under the compliance: block in config.yaml.

Option	Type	Default	Description
`enabled`	`bool`	`false`	Master switch. All other options are ignored when `false`.
`server_url`	`str`	`https://www.aissurance.eu/api/v1/discovery/receive`	Discovery receive URL. Must be HTTPS. Health and config-report URLs are derived from it.
`api_key`	`str`	`${AISSURANCE_KEY}`	Tenant secret. Used as the Bearer token and as the HMAC signing key.
`tenant_id`	`str`	`${AISSURANCE_TENANT_ID}`	Tenant UUID. Optional if embedded in the key.
`router_id`	`str`	machine hostname	Unique identifier for this Router instance. Multiple instances per tenant are allowed.
`polling_interval`	`int`	`300`	Seconds between discovery snapshots. Minimum `60`.
`health_interval`	`int`	`3600`	Seconds between health / keepalive calls.
`batch_size`	`int`	`50`	Max models per payload. On `413` the payload is auto-split into batches.
`max_retry_attempts`	`int`	`10`	In-cycle send retries before the payload is buffered to disk.
`retry_backoff_base`	`int`	`2`	Exponential backoff base in seconds (`base ** attempt`, capped at 60s).
`buffer_dir`	`str`	`./compliance-buffer`	Directory for encrypted offline retry files.
`max_buffer_payloads`	`int`	`100`	Buffer cap. Oldest file is dropped first when exceeded.

`polling_interval`

How often a snapshot is collected and sent. The server enforces a maximum of one payload per polling_interval per tenant; sending faster yields 429. Values below 60 are rejected at startup.

`batch_size`

Caps how many models a single payload carries. If the upstream replies 413 Payload Too Large, the plugin splits the model list into batch_size-sized chunks, re-signs each chunk, and sends them individually.

`buffer_dir` and `max_buffer_payloads`

When the upstream is unreachable, each signed payload is encrypted and written to buffer_dir. The directory is bounded by max_buffer_payloads; once full, the oldest file is dropped before a new one is written. Buffered payloads are replayed oldest-first on the next successful cycle. Add buffer_dir to your backup exclusions — files are encrypted but transient.

Environment variables

Variable	Required	Description
`AISSURANCE_KEY`	Yes	Tenant secret issued at signup. Referenced by `api_key: "${AISSURANCE_KEY}"`.
`AISSURANCE_HMAC_SECRET`	No	Per-tenant HMAC secret override. When set, it signs payloads instead of `AISSURANCE_KEY`.
`AISSURANCE_TENANT_ID`	No	Tenant UUID, if not embedded in the key.
`AISSURANCE_VERIFY_TLS`	No	Set to `"0"` to allow self-signed certs (development only — logged as a warning).

Values referenced as ${VAR} in config.yaml are expanded at load time. If a referenced variable is unset, the corresponding environment-variable default above takes over.

Outbound endpoints

All three are Router-initiated POSTs. URLs are derived from server_url by replacing the trailing /receive path.

Endpoint	When called	Purpose
`POST …/discovery/receive`	every `polling_interval`	Send the signed discovery payload.
`POST …/discovery/health/{tenant_id}`	on startup + every `health_interval`	Connectivity check / keepalive / config sync.
`POST …/discovery/config/report`	on startup handshake	Report the active compliance config for acknowledgment.

Every request carries:

Authorization: Bearer ${AISSURANCE_KEY}
X-Router-Version: <router version>
X-Router-Id: <router_id>
Content-Type: application/json

Response handling

Status	Meaning	Plugin action
`202` / `200`	Accepted	Mark sent; replay any buffered payloads.
`401` / `403`	Invalid key / blacklisted	Stop reporting, clear the buffer, log `auth_failed`.
`429`	Rate limited	Drop this snapshot (the next one is just as fresh).
`413`	Too large	Split by `batch_size`, re-sign, resend.
`5xx` / network error	Server / connectivity failure	Retry with backoff, then encrypt + buffer to disk.

Payload structure

{
  "tenant_id": "550e8400-e29b-41d4-a716-446655440000",
  "router_id": "router-prod-001",
  "router_version": "0.9",
  "timestamp": "2026-06-10T10:30:00Z",
  "models": [
    {
      "name": "mistral-7b-instruct",
      "version": "latest",
      "quantization": "Q4_K_M",
      "size_gb": 4.1,
      "endpoint": "http://192.168.0.50:11434",
      "last_used": "2026-06-10T10:28:14Z",
      "request_count_24h": null,
      "avg_latency_ms": null,
      "p99_latency_ms": null,
      "error_count_24h": null,
      "cache_hit_rate": null,
      "avg_input_tokens": null,
      "avg_output_tokens": null
    }
  ],
  "endpoints": [
    {
      "url": "http://192.168.0.50:11434",
      "type": "ollama",
      "status": "healthy",
      "concurrent_connections": 2,
      "max_concurrent_connections": 4,
      "tls_enabled": false,
      "auth_method": "none"
    }
  ],
  "telemetry": {
    "total_requests_24h": null,
    "total_cache_hits": 1547,
    "cache_hit_rate": 0.634,
    "error_rate": null,
    "total_bytes_in_24h": 9120344,
    "total_bytes_out_24h": 14882910,
    "active_connections": 2,
    "uptime_hours": 72.4
  },
  "hmac_signature": "sha256=abcdef…"
}

endpoint.type is best-effort: ollama, llama_cpp, vllm, openai, gemini, anthropic, cohere, cerebras, inception labs, or other. endpoint.status maps the Router's health probe onto healthy / unhealthy.

Security

Dual-key model

Credential	Protects	Managed by
`nomyo-router-api-key`	The Router's own API / dashboard	Router operator
`AISSURANCE_KEY`	The upstream reporting channel	aissurance.eu (issued at signup)

These are independent. See Router API key usage for the former.

Payload signing

Every payload is HMAC-SHA256 signed. The signature covers all fields except hmac_signature, over a canonical sort_keys serialization, so any tampering with the body invalidates it. The signing key is AISSURANCE_HMAC_SECRET if set, otherwise AISSURANCE_KEY.

Buffer encryption

Offline buffer files are encrypted at rest with a key derived from your AISSURANCE_KEY via HKDF-SHA256 (Fernet / AES-128-CBC + HMAC). A leaked buffer file is useless without the tenant secret. Corrupt or wrong-key files are dropped on read.

Transport

TLS is required (server_url must be HTTPS). AISSURANCE_VERIFY_TLS=0 disables certificate verification for development only and is logged as a warning.

Observability

The plugin writes structured JSON logs to stderr (one object per line):

{"timestamp":"2026-06-10T10:30:00Z","level":"info","module":"aissurance","event":"payload_sent","tenant_id":"abc123","router_id":"router-prod-001","models_count":12,"response_code":202,"latency_ms":245}

Event types:

Event	Meaning
`plugin_started` / `plugin_stopped`	Lifecycle. `plugin_stopped` reports buffered + successful counts.
`plugin_disabled`	Startup validation failed (e.g. missing key, non-HTTPS URL). Reasons included.
`payload_sent`	A discovery payload (or a replayed buffer file) was accepted.
`payload_buffered`	Upstream unreachable; payload encrypted to disk.
`payload_retry`	A transient failure is being retried with backoff.
`payload_rate_limited`	Server returned `429`; snapshot dropped.
`auth_failed`	Key rejected (`401`/`403`); reporting halted, buffer cleared. Operator action required.
`config_reported`	Config handshake completed; any server suggestion included.
`snapshot_too_slow`	Collection exceeded its 10s budget; cycle skipped.
`tenant_id_missing`	No tenant id resolved; payloads will carry `tenant_id: null`.

Lifecycle

Startup — validate config (HTTPS URL, key present, polling_interval ≥ 60); on failure the plugin logs plugin_disabled and stays dormant without affecting the Router. On success it performs a health + config-report handshake, then starts the snapshot and health loops.

Runtime — the first snapshot is emitted immediately so evidence appears without waiting a full interval, then the plugin settles into polling_interval. Snapshot collection runs under a 10s budget and reads from shared Router state without blocking request routing.

Shutdown — the loops stop, the HTTP client closes, and a plugin_stopped log records how many payloads were buffered vs. sent. The Router continues and completes its own shutdown normally.

Config changes require a restart. Like the rest of the Router config, the compliance block is read at startup. Hot-reload of the compliance settings is not yet supported.

Troubleshooting

Symptom	Likely cause	Fix
`plugin_disabled` at startup	Missing key, non-HTTPS `server_url`, or `polling_interval < 60`	Check the `reasons` array in the log line.
`auth_failed`, reporting stops	`AISSURANCE_KEY` invalid/expired/blacklisted	Re-provision the key on aissurance.eu and restart.
Buffer files accumulating	Upstream unreachable	Confirm egress to aissurance.eu over HTTPS; check firewall/NAT.
`tenant_id` is `null` in payloads	No `AISSURANCE_TENANT_ID` / `compliance.tenant_id`	Set it, unless your key embeds the tenant.
`snapshot_too_slow` warnings	Slow/unhealthy backends delaying health probes	Investigate endpoint health; the cycle self-recovers next tick.

13 KiB Raw Permalink Blame History