mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-04-27 01:16:22 +02:00
feat: IAM service, gateway auth middleware, capability model, and CLIs (#849)
Replaces the legacy GATEWAY_SECRET shared-token gate with an IAM-backed
identity and authorisation model. The gateway no longer has an
"allow-all" or "no auth" mode; every request is authenticated via the
IAM service, authorised against a capability model that encodes both
the operation and the workspace it targets, and rejected with a
deliberately-uninformative 401 / 403 on any failure.
IAM service (trustgraph-flow/trustgraph/iam, trustgraph-base/schema/iam)
-----------------------------------------------------------------------
* New backend service (iam-svc) owning users, workspaces, API keys,
passwords and JWT signing keys in Cassandra. Reached over the
standard pub/sub request/response pattern; gateway is the only
caller.
* Operations: bootstrap, resolve-api-key, login, get-signing-key-public,
rotate-signing-key, create/list/get/update/disable/delete/enable-user,
change-password, reset-password, create/list/get/update/disable-
workspace, create/list/revoke-api-key.
* Ed25519 JWT signing (alg=EdDSA). Key rotation writes a new kid and
retires the previous one; validation is grace-period friendly.
* Passwords: PBKDF2-HMAC-SHA-256, 600k iterations, per-user salt.
* API keys: 128-bit random, SHA-256 hashed. Plaintext returned once.
* Bootstrap is explicit: --bootstrap-mode {token,bootstrap} is a
required startup argument with no permissive default. Masked
"auth failure" errors hide whether a refused bootstrap request was
due to mode, state, or authorisation.
Gateway authentication (trustgraph-flow/trustgraph/gateway/auth.py)
-------------------------------------------------------------------
* IamAuth replaces the legacy Authenticator. Distinguishes JWTs
(three-segment dotted) from API keys by shape; verifies JWTs
locally using the cached IAM public key; resolves API keys via
IAM with a short-TTL hash-keyed cache. Every failure path
surfaces the same 401 body ("auth failure") so callers cannot
enumerate credential state.
* Public key is fetched at gateway startup with a bounded retry loop;
traffic does not begin flowing until auth has started.
Capability model (trustgraph-flow/trustgraph/gateway/capabilities.py)
---------------------------------------------------------------------
* Roles have two dimensions: a capability set and a workspace scope.
OSS ships reader / writer / admin; the first two are workspace-
assigned, admin is cross-workspace ("*"). No "cross-workspace"
pseudo-capability — workspace permission is a property of the role.
* check(identity, capability, target_workspace=None) is the single
authorisation test: some role must grant the capability *and* be
active in the target workspace.
* enforce_workspace validates a request-body workspace against the
caller's role scopes and injects the resolved value. Cross-
workspace admin is permitted by role scope, not by a bypass.
* Gateway endpoints declare a required capability explicitly — no
permissive default. Construction fails fast if omitted. Enterprise
editions can replace the role table without changing the wire
protocol.
WebSocket first-frame auth (dispatch/mux.py, endpoint/socket.py)
----------------------------------------------------------------
* /api/v1/socket handshake unconditionally accepts; authentication
runs on the first WebSocket frame ({"type":"auth","token":"..."})
with {"type":"auth-ok","workspace":"..."} / {"type":"auth-failed"}.
The socket stays open on failure so the client can re-authenticate
— browsers treat a handshake-time 401 as terminal, breaking
reconnection.
* Mux.receive rejects every non-auth frame before auth succeeds,
enforces the caller's workspace (envelope + inner payload) using
the role-scope resolver, and supports mid-session re-auth.
* Flow import/export streaming endpoints keep the legacy ?token=
handshake (URL-scoped short-lived transfers; no re-auth need).
Auth surface
------------
* POST /api/v1/auth/login — public, returns a JWT.
* POST /api/v1/auth/bootstrap — public; forwards to IAM's bootstrap
op which itself enforces mode + tables-empty.
* POST /api/v1/auth/change-password — any authenticated user.
* POST /api/v1/iam — admin-only generic forwarder for the rest of
the IAM API (per-op REST endpoints to follow in a later change).
Removed / breaking
------------------
* GATEWAY_SECRET / --api-token / default_api_token and the legacy
Authenticator.permitted contract. The gateway cannot run without
IAM.
* ?token= on /api/v1/socket.
* DispatcherManager and Mux both raise on auth=None — no silent
downgrade path.
CLI tools (trustgraph-cli)
--------------------------
tg-bootstrap-iam, tg-login, tg-create-user, tg-list-users,
tg-disable-user, tg-enable-user, tg-delete-user, tg-change-password,
tg-reset-password, tg-create-api-key, tg-list-api-keys,
tg-revoke-api-key, tg-create-workspace, tg-list-workspaces. Passwords
read via getpass; tokens / one-time secrets written to stdout with
operator context on stderr so shell composition works cleanly.
AsyncSocketClient / SocketClient updated to the first-frame auth
protocol.
Specifications
--------------
* docs/tech-specs/iam.md updated with the error policy, workspace
resolver extension point, and OSS role-scope model.
* docs/tech-specs/iam-protocol.md (new) — transport, dataclasses,
operation table, error taxonomy, bootstrap modes.
* docs/tech-specs/capabilities.md (new) — capability vocabulary, OSS
role bundles, agent-as-composition note, enforcement-boundary
policy, enterprise extensibility.
Tests
-----
* test_auth.py (rewritten) — IamAuth + JWT round-trip with real
Ed25519 keypairs + API-key cache behaviour.
* test_capabilities.py (new) — role table sanity, check across
role x workspace combinations, enforce_workspace paths,
unknown-cap / unknown-role fail-closed.
* Every endpoint test construction now names its capability
explicitly (no permissive defaults relied upon). New tests pin
the fail-closed invariants: DispatcherManager / Mux refuse
auth=None; i18n path-traversal defense is exercised.
* test_socket_graceful_shutdown rewritten against IamAuth.
This commit is contained in:
cybermaggedon
GitHub
parent
ae9936c9cc
commit
67b2fc448f
61 changed files with 6474 additions and 792 deletions
218
docs/tech-specs/capabilities.md
Normal file
218
docs/tech-specs/capabilities.md
Normal file
|
|
@ -0,0 +1,218 @@
|
|||
---
|
||||
layout: default
|
||||
title: "Capability Vocabulary Technical Specification"
|
||||
parent: "Tech Specs"
|
||||
---
|
||||
|
||||
# Capability Vocabulary Technical Specification
|
||||
|
||||
## Overview
|
||||
|
||||
Authorisation in TrustGraph is **capability-based**. Every gateway
|
||||
endpoint maps to exactly one *capability*; a user's roles each grant
|
||||
a set of capabilities; an authenticated request is permitted when
|
||||
the required capability is a member of the union of the caller's
|
||||
role capability sets.
|
||||
|
||||
This document defines the capability vocabulary — the closed list
|
||||
of capability strings that the gateway recognises — and the
|
||||
open-source edition's role bundles.
|
||||
|
||||
The capability mechanism is shared between open-source and potential
|
||||
3rd party enterprise capability. The open-source edition ships a
|
||||
fixed three-role bundle (`reader`, `writer`, `admin`). Enterprise
|
||||
capability may define additional roles by composing their own
|
||||
capability bundles from the same vocabulary; no protocol, gateway,
|
||||
or backend-service change is required.
|
||||
|
||||
## Motivation
|
||||
|
||||
The original IAM spec used hierarchical "minimum role" checks
|
||||
(`admin` implies `writer` implies `reader`). That shape is simple
|
||||
but paints the role model into a corner: any enterprise need to
|
||||
grant a subset of admin abilities (helpdesk that can reset
|
||||
passwords but not edit flows; analyst who can query but not ingest)
|
||||
requires a protocol-level change.
|
||||
|
||||
A capability vocabulary decouples "what a request needs" from
|
||||
"what roles a user has" and makes the role table pure data. The
|
||||
open-source bundles can stay coarse while the enterprise role
|
||||
table expands without any code movement.
|
||||
|
||||
## Design
|
||||
|
||||
### Capability string format
|
||||
|
||||
`<subsystem>:<verb>` or `<subsystem>` (for capabilities with no
|
||||
natural read/write split). All lowercase, kebab-case for
|
||||
multi-word subsystems.
|
||||
|
||||
### Capability list
|
||||
|
||||
**Data plane**
|
||||
|
||||
| Capability | Covers |
|
||||
|---|---|
|
||||
| `agent` | agent (query-only; no write counterpart) |
|
||||
| `graph:read` | graph-rag, graph-embeddings-query, triples-query, sparql, graph-embeddings-export, triples-export |
|
||||
| `graph:write` | triples-import, graph-embeddings-import |
|
||||
| `documents:read` | document-rag, document-embeddings-query, document-embeddings-export, entity-contexts-export, document-stream-export, library list / fetch |
|
||||
| `documents:write` | document-embeddings-import, entity-contexts-import, text-load, document-load, library add / replace / delete |
|
||||
| `rows:read` | rows-query, row-embeddings-query, nlp-query, structured-query, structured-diag |
|
||||
| `rows:write` | rows-import |
|
||||
| `llm` | text-completion, prompt (stateless invocation) |
|
||||
| `embeddings` | Raw text-embedding service (stateless compute; typed-data embedding stores live under their data-subject capability) |
|
||||
| `mcp` | mcp-tool |
|
||||
| `collections:read` | List / describe collections |
|
||||
| `collections:write` | Create / delete collections |
|
||||
| `knowledge:read` | List / get knowledge cores |
|
||||
| `knowledge:write` | Create / delete knowledge cores |
|
||||
|
||||
**Control plane**
|
||||
|
||||
| Capability | Covers |
|
||||
|---|---|
|
||||
| `config:read` | Read workspace config |
|
||||
| `config:write` | Write workspace config |
|
||||
| `flows:read` | List / describe flows, blueprints, flow classes |
|
||||
| `flows:write` | Start / stop / update flows |
|
||||
| `users:read` | List / get users within the workspace |
|
||||
| `users:write` | Create / update / disable users within the workspace |
|
||||
| `users:admin` | Assign / remove roles on users within the workspace |
|
||||
| `keys:self` | Create / revoke / list **own** API keys |
|
||||
| `keys:admin` | Create / revoke / list **any user's** API keys within the workspace |
|
||||
| `workspaces:admin` | Create / delete / disable workspaces (system-level) |
|
||||
| `iam:admin` | JWT signing-key rotation, IAM-level operations |
|
||||
| `metrics:read` | Prometheus metrics proxy |
|
||||
|
||||
### Open-source role bundles
|
||||
|
||||
The open-source edition ships three roles:
|
||||
|
||||
| Role | Capabilities |
|
||||
|---|---|
|
||||
| `reader` | `agent`, `graph:read`, `documents:read`, `rows:read`, `llm`, `embeddings`, `mcp`, `collections:read`, `knowledge:read`, `flows:read`, `config:read`, `keys:self` |
|
||||
| `writer` | everything in `reader` **+** `graph:write`, `documents:write`, `rows:write`, `collections:write`, `knowledge:write` |
|
||||
| `admin` | everything in `writer` **+** `config:write`, `flows:write`, `users:read`, `users:write`, `users:admin`, `keys:admin`, `workspaces:admin`, `iam:admin`, `metrics:read` |
|
||||
|
||||
Open-source bundles are deliberately coarse. `workspaces:admin` and
|
||||
`iam:admin` live inside `admin` without a separate role; a single
|
||||
`admin` user holds the keys to the whole deployment.
|
||||
|
||||
### The `agent` capability and composition
|
||||
|
||||
The `agent` capability is granted independently of the capabilities
|
||||
it composes under the hood (`llm`, `graph`, `documents`, `rows`,
|
||||
`mcp`, etc.). A user holding `agent` but not `llm` can still cause
|
||||
LLM invocations because the agent implementation chooses which
|
||||
services to invoke on the caller's behalf.
|
||||
|
||||
This is deliberate. A common policy is "allow controlled access
|
||||
via the agent, deny raw model calls" — granting `agent` without
|
||||
granting `llm` expresses exactly that. An administrator granting
|
||||
`agent` should treat it as a grant of everything the agent
|
||||
composes at deployment time.
|
||||
|
||||
### Authorisation evaluation
|
||||
|
||||
For a request bearing a resolved set of roles
|
||||
`R = {r1, r2, ...}` against an endpoint that requires capability
|
||||
`c`:
|
||||
|
||||
```
|
||||
allow if c IN union(bundle(r) for r in R)
|
||||
```
|
||||
|
||||
No hierarchy, no precedence, no role-order sensitivity. A user
|
||||
with a single role is the common case; a user with multiple roles
|
||||
gets the union of their bundles.
|
||||
|
||||
### Enforcement boundary
|
||||
|
||||
Capability checks — and authentication — are applied **only at the
|
||||
API gateway**, on requests arriving from external callers.
|
||||
Operations originating inside the platform (backend service to
|
||||
backend service, agent to LLM, flow-svc to config-svc, bootstrap
|
||||
initialisers, scheduled reconcilers, autonomous flow steps) are
|
||||
**not capability-checked**. Backend services trust the workspace
|
||||
set by the gateway on inbound pub/sub messages and trust
|
||||
internally-originated messages without further authorisation.
|
||||
|
||||
This policy has four consequences that are part of the spec, not
|
||||
accidents of implementation:
|
||||
|
||||
1. **The gateway is the single trust boundary for user
|
||||
authorisation.** Every backend service is a downstream consumer
|
||||
of an already-authorised workspace scope.
|
||||
2. **Pub/sub carries workspace, not user identity.** Messages on
|
||||
the bus do not carry credentials or the identity that originated
|
||||
a request; they carry the resolved workspace only. This keeps
|
||||
the bus protocol free of secrets and aligns with the workspace
|
||||
resolver's role as the gateway-side narrowing step.
|
||||
3. **Composition is transitive.** Granting a capability that the
|
||||
platform composes internally (for example, `agent`) transitively
|
||||
grants everything that capability composes under the hood,
|
||||
because the downstream calls are internal-origin and are not
|
||||
re-checked. The composite nature of `agent` described above is
|
||||
a consequence of this policy, not a special case.
|
||||
4. **Internal-origin operations have no user.** Bootstrap,
|
||||
reconcilers, and other platform-initiated work act with
|
||||
system-level authority. The workspace field on such messages
|
||||
identifies which workspace's data is being touched, not who
|
||||
asked.
|
||||
|
||||
**Trust model.** Whoever has pub/sub access is implicitly trusted
|
||||
to act as any workspace. Defense-in-depth within the backend is
|
||||
not part of this design; the security perimeter is the gateway
|
||||
and the bus itself (TLS / network isolation between the bus and
|
||||
any untrusted network).
|
||||
|
||||
### Unknown capabilities and unknown roles
|
||||
|
||||
- An endpoint declaring an unknown capability is a server-side bug
|
||||
and fails closed (403, logged).
|
||||
- A user carrying a role name that is not defined in the role table
|
||||
is ignored for authorisation purposes and logged as a warning.
|
||||
Behaviour is deterministic: unknown roles contribute zero
|
||||
capabilities.
|
||||
|
||||
### Capability scope
|
||||
|
||||
Every capability is **implicitly scoped to the caller's resolved
|
||||
workspace**. A `users:write` capability does not permit a user
|
||||
in workspace `acme` to create users in workspace `beta` — the
|
||||
workspace-resolver has already narrowed the request to one
|
||||
workspace before the capability check runs. See the IAM
|
||||
specification for the workspace-resolver contract.
|
||||
|
||||
The three exceptions are the system-level capabilities
|
||||
`workspaces:admin` and `iam:admin`, which operate across
|
||||
workspaces by definition, and `metrics:read`, which returns
|
||||
process-level series not scoped to any workspace.
|
||||
|
||||
## Enterprise extensibility
|
||||
|
||||
Enterprise editions extend the role table additively:
|
||||
|
||||
```
|
||||
data-analyst: {query, library:read, collections:read, knowledge:read}
|
||||
helpdesk: {users:read, users:write, users:admin, keys:admin}
|
||||
data-engineer: writer + {flows:read, config:read}
|
||||
workspace-owner: admin − {workspaces:admin, iam:admin}
|
||||
```
|
||||
|
||||
None of this requires a protocol change — the wire-protocol `roles`
|
||||
field on user records is already a set, the gateway's
|
||||
capability-check is already capability-based, and the capability
|
||||
vocabulary is closed. Enterprises may introduce roles whose bundles
|
||||
compose the same capabilities differently.
|
||||
|
||||
When an enterprise introduces a new capability (e.g. for a feature
|
||||
that does not exist in open source), the capability string is
|
||||
added to the vocabulary and recognised by the gateway build that
|
||||
ships that feature.
|
||||
|
||||
## References
|
||||
|
||||
- [Identity and Access Management Specification](iam.md)
|
||||
- [Architecture Principles](architecture-principles.md)
|
||||
Loading…
Add table
Add a link
Reference in a new issue