mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-06-09 01:35:18 +02:00
* feat(MR-656): inline query strings in CLI and HTTP server
CLI:
- Add -e / --query-string <STRING> to omnigraph read and omnigraph change
- Exactly one of --query, --query-string, --alias is required (3-way XOR)
- Empty --query-string is rejected with a clear error
HTTP:
- New POST /query (read-only, clean field names: query/name/params/branch/snapshot)
- Mutations on /query are rejected with 400 -- use POST /change instead
- ChangeRequest fields polished: query (alias query_source), name (alias query_name)
- POST /read and POST /change remain byte-compatible for existing clients
Tests:
- cli.rs: -e happy-path on read/change, mutex error vs --query, empty -e rejected
- system_local.rs: inline -e read and -e change exercise the local flow
- system_remote.rs: inline -e read/change over HTTP plus direct /query 200/400
- server.rs: /query 200, /query 400 on mutation, /change legacy field alias
- openapi.rs: new /query path, QueryRequest schema, ChangeRequest field-name polish
Docs: cli.md (-e examples), cli-reference.md (read/change rows), server.md (/query)
Co-Authored-By: Ragnor Comerford <ragnor.comerford@gmail.com>
* feat(MR-656): rename read/change to query/mutate with deprecation signals
HTTP server:
- Add POST /mutate as canonical write endpoint (pairs with POST /query).
- Mark POST /read and POST /change as deprecated. Three-channel signal:
* OpenAPI: `deprecated: true` on the operation (every codegen flags
the generated SDK method).
* RFC 9745: response `Deprecation: true` header on every response.
* RFC 8288: response `Link: </successor>; rel="successor-version"`
pointing at /query and /mutate respectively.
- Share business logic across /mutate and /change via run_mutate(); the
/change wrapper is the only place that adds the deprecation headers.
- ChangeRequest field aliases (query_source/query_name) preserved.
- AliasCommand serde now accepts `query`/`mutate` alongside `read`/`change`.
CLI:
- Promote `omnigraph query` / `omnigraph mutate` to top-level canonical
subcommands (clap visible_alias keeps `omnigraph read` / `omnigraph
change` working forever).
- Promote `omnigraph lint` / `omnigraph check` to top-level (was nested
under `omnigraph query lint`, which is now a deprecated argv shim that
rewrites to the canonical form).
- Argv-level preprocessing prints a one-line deprecation warning to
stderr when any legacy spelling is used. Canonical names are silent.
Tests:
- Server: /mutate works, /change emits Deprecation+Link headers, /read
emits Deprecation+Link headers, /query carries no deprecation signal.
- OpenAPI: /read and /change flagged deprecated; /query and /mutate not.
- CLI: canonical `lint` matches deprecated `query lint` / `query check`
output; `read` / `change` print deprecation warnings.
Docs:
- cli.md: new canonical examples; "Deprecated names" migration table.
- cli-reference.md: top-level table updated; aliases.<name>.command
accepts both legacy and canonical spellings.
- server.md: endpoint inventory shows /query and /mutate as canonical
and /read and /change as deprecated; dedicated section explains the
three-channel deprecation signal.
- og-cheet-sheet.md: use new `omnigraph lint` / `omnigraph check`.
- openapi.json regenerated.
Migration is purely cosmetic — every deprecated form continues to work
indefinitely; only the spelling changes.
Co-Authored-By: Ragnor Comerford <ragnor.comerford@gmail.com>
* fix(MR-656): address Devin Review findings on /query and /change
Two issues raised by Devin Review on PR #110:
1. `POST /query` mutation-rejection error pointed at the deprecated
`/change` endpoint instead of the canonical `/mutate`. Fixed in
three places: the runtime error message in `server_query`, the
utoipa 400-response description, and the handler doc comment. The
`QueryRequest` schema docstrings in `api.rs` got the same update so
the openapi.json bodies match. Server and openapi tests updated.
2. `execute_change_remote` serialized `ChangeRequest` directly, which
emits the new canonical field names `query` / `name` on the wire.
`#[serde(alias = "query_source")]` only affects deserialization, so
a newer CLI talking to an older server would have its `/change`
POST body fail with "missing field: query_source". Fixed by
extracting a `legacy_change_request_body` helper that hand-rolls
the JSON with the legacy keys (`query_source` / `query_name`), the
same byte-stable contract `execute_read_remote` already uses
against `/read`. Added two unit tests on the helper to lock the
wire shape in.
Co-Authored-By: Ragnor Comerford <ragnor.comerford@gmail.com>
* docs(dev): RFC 001 — inline + stored queries, envelope, MCP
Tracked artifact consolidating the design across MR-656 (this branch),
MR-976 (Phase 1 envelope hardening parent, with MR-977/978/979/980
sub-issues), and MR-969 (stored queries + MCP).
Sections:
* Two paths, one engine — inline `/query` + `/mutate` (this PR) coexist
with stored `/queries/{name}` (MR-969). Same `run_query` / `run_mutate`
backend (the fold-in landed in the previous commit).
* Request envelope ("before") — Idempotency-Key, If-Match, X-Deadline,
X-Trace-Id, expect, dry_run, fields. Phase 1 ships the load-bearing
subset on `/mutate`.
* Response envelope ("after") — audit_id, snapshot_id, commit_id, stats,
warnings. Closes the provenance loop today's `ChangeOutput` leaves
open.
* `.gq` pragmas — `@description`, `@returns`, `@mcp`. Source-of-truth
for the stored-query agent contract; no separate YAML registry.
* Multi-graph MCP — per-graph `/graphs/{id}/mcp/tools` + `/mcp/invoke`.
Token binds to one graph by default; cross-graph agents loop.
* Cedar split — `read`/`change` for inline, `invoke_query` for stored.
Operators deny ad-hoc for agent groups while keeping curated tool
list open.
* Rejected alternatives — per-env override files, compiled bundles,
tool-name prefixing across graphs, body-field graph dispatch.
Index entry added under "Active Implementation Plans" so future agents
land on the RFC before touching queries / mutations / envelope code.
`scripts/check-agents-md.sh` clean (35 links, 34 docs).
* docs(server): clarify why run_query lacks AppState parameter
run_mutate takes state for workload admission; run_query doesn't because
reads aren't admission-gated today. Mark the asymmetry as intentional and
flag the two future events that would grow the signature: Phase 1's
`expect: { max_rows_scanned: N }` budget (MR-976) or per-actor admission
extending to stored-read invocations (MR-969). Prevents the natural
"make these symmetrical" follow-up.
* refactor(server): run_query / run_mutate take &ResolvedActor
Replace `Option<Extension<ResolvedActor>>` in the helpers with
`Option<&ResolvedActor>`. Saves MR-969's stored-query handler from
wrapping a bare actor in axum's `Extension(...)` before calling.
Handler signatures (`server_query`, `server_read`, `server_mutate`,
`server_change`) keep `Option<Extension<ResolvedActor>>` because that
is what axum injects, and unwrap at the call site with
`actor.as_ref().map(|Extension(actor)| actor)`.
Net: -13/+10 LOC, 89/0 server tests pass.
* docs(releases): v0.6.0 — describe inline + canonical-named queries (MR-656)
Extend the v0.6.0 release notes to cover the third piece of work landing
alongside the graph terminology rename and multi-graph server mode:
canonical-named `POST /query` and `POST /mutate` endpoints, the CLI's
new `-e/--query-string` flag, the top-level promotion of `lint` /
`check`, and the three-channel deprecation signal on `/read` and
`/change` (OpenAPI `deprecated: true` + RFC 9745 + RFC 8288).
Additions:
* Top blurb: "Two pieces" -> "Three pieces" with a bullet describing
the rename + inline flow.
* Breaking Changes: new "Query / mutation rename" subsection covering
the `ChangeRequest` field rename (with the back-compat serde aliases
and the CLI's `legacy_change_request_body` byte-stable wire helper)
and the `omnigraph query lint` -> `omnigraph lint` move.
* New: 5 bullets — the two endpoints, the CLI subcommands, the `-e`
flag, the deprecation signal channels, the widened `aliases.<name>.command`
vocabulary.
* User Impact: one bullet making explicit that the rename is cosmetic
on the client side and migration is voluntary.
* Documentation: pointers to the updated `server.md` / `cli.md` /
`cli-reference.md` and the new `docs/dev/rfc-001-queries-envelope-mcp.md`.
+15/-1 lines. `./scripts/check-agents-md.sh` clean.
* refactor(cli): demote `check` from visible_alias to deprecation shim
`omnigraph check` was a clap `visible_alias` on `lint`, advertised in
`--help` as an equivalent canonical name. Per MR-981 §6 (long-form
flags as canonical, short forms as visible aliases), visible aliases
on subcommand names hurt agent CX: agents emit either spelling
depending on training-data drift, and there's no length signal
pointing at the canonical name.
Changes:
* Remove `#[command(visible_alias = "check")]` from the `Lint` variant.
`omnigraph --help` now shows only `lint`.
* Add bare `check` to `rewrite_deprecated_argv` so `omnigraph check
<args>` still works — it rewrites to `omnigraph lint <args>` and
emits a one-line stderr deprecation warning, matching the existing
pattern for `read` / `change` / `query lint` / `query check`.
* Fix the nested `query check` shim to substitute `check` -> `lint` in
the rewritten argv (previously it relied on `check` being a
visible_alias to reach the `Lint` variant).
* New test `deprecated_check_top_level_rewrites_to_lint` covers: bare
`check` produces identical stdout to `lint`, emits the deprecation
warning, and `check` does NOT appear as an alias in `omnigraph
--help`.
* Release notes updated to reflect the deprecation-shim treatment and
cross-reference MR-981 §6 reasoning.
Cargo / Go users typing `check` still work indefinitely; one stderr
nudge per invocation teaches the canonical name. Agents see only
`lint` in `--help --json` so they emit one canonical form.
67/0 omnigraph-cli tests pass; 39 workspace test suites green.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ragnor Comerford <ragnor.comerford@gmail.com>
Co-authored-by: Ragnor Comerford <hello@ragnor.co>
192 lines
10 KiB
Markdown
192 lines
10 KiB
Markdown
# HTTP Server (`omnigraph-server`)
|
|
|
|
Axum 0.8 + tokio + utoipa-generated OpenAPI. **Two modes** (v0.6.0+): single-graph (legacy) and multi-graph (MR-668). Mode is inferred from CLI args + config shape.
|
|
|
|
## Modes
|
|
|
|
### Single-graph mode (legacy)
|
|
|
|
`omnigraph-server <URI>` or `omnigraph-server --target <name> --config omnigraph.yaml`. Routes are flat — `/snapshot`, `/read`, `/branches`, etc. Behavior unchanged from v0.6.0.
|
|
|
|
### Multi-graph mode (v0.6.0+)
|
|
|
|
`omnigraph-server --config omnigraph.yaml` with a non-empty `graphs:` map and **no** single-mode selector (no `server.graph`, no `<URI>`, no `--target`). The server opens every configured graph in parallel at startup (bounded concurrency = 4, fail-fast on the first open error). Routes are nested under `/graphs/{graph_id}/...`. Bare flat paths return 404 in multi mode.
|
|
|
|
Mode inference (four-rule matrix):
|
|
|
|
1. CLI positional `<URI>` → single
|
|
2. CLI `--target <name>` → single
|
|
3. `server.graph` in config → single
|
|
4. `--config` + non-empty `graphs:` + no single-mode selector → **multi**
|
|
5. otherwise → error with migration hint
|
|
|
|
## Endpoint inventory
|
|
|
|
Per-graph endpoints — same body shape across modes; URLs differ:
|
|
|
|
| Method | Single-mode path | Multi-mode path | Auth | Action | Handler |
|
|
|---|---|---|---|---|---|
|
|
| GET | `/healthz` | `/healthz` | none | — | `server_health` |
|
|
| GET | `/openapi.json` | `/openapi.json` | none | — | `server_openapi` (strips security if auth disabled; in multi mode emits cluster paths with `cluster_` operation-id prefix) |
|
|
| GET | `/snapshot?branch=` | `/graphs/{id}/snapshot?branch=` | bearer + `read` | snapshot of branch | `server_snapshot` |
|
|
| POST | `/query` | `/graphs/{id}/query` | bearer + `read` | inline read query (canonical; clean field names `query`/`name`; mutations → 400) | `server_query` |
|
|
| POST | `/read` | `/graphs/{id}/read` | bearer + `read` | **deprecated** alias of `/query` (legacy field names `query_source`/`query_name`, byte-stable response; carries `Deprecation: true` + `Link: </query>; rel="successor-version"`) | `server_read` |
|
|
| POST | `/export` | `/graphs/{id}/export` | bearer + `export` | NDJSON stream | `server_export` |
|
|
| POST | `/mutate` | `/graphs/{id}/mutate` | bearer + `change` | mutation (canonical; `query`/`name`; accepts legacy `query_source`/`query_name` as serde aliases) | `server_mutate` |
|
|
| POST | `/change` | `/graphs/{id}/change` | bearer + `change` | **deprecated** alias of `/mutate` (carries `Deprecation: true` + `Link: </mutate>; rel="successor-version"`) | `server_change` |
|
|
| GET | `/schema` | `/graphs/{id}/schema` | bearer + `read` | get current `.pg` source | `server_schema_get` |
|
|
| POST | `/schema/apply` | `/graphs/{id}/schema/apply` | bearer + `schema_apply` (target=`main`) | migrate | `server_schema_apply` |
|
|
| POST | `/ingest` | `/graphs/{id}/ingest` | bearer + `branch_create` (if new) + `change` | bulk load | `server_ingest` (32 MB body limit) |
|
|
| GET | `/branches` | `/graphs/{id}/branches` | bearer + `read` | list branches | `server_branch_list` |
|
|
| POST | `/branches` | `/graphs/{id}/branches` | bearer + `branch_create` | create | `server_branch_create` |
|
|
| DELETE | `/branches/{branch}` | `/graphs/{id}/branches/{branch}` | bearer + `branch_delete` | delete | `server_branch_delete` |
|
|
| POST | `/branches/merge` | `/graphs/{id}/branches/merge` | bearer + `branch_merge` | merge `source → target` | `server_branch_merge` |
|
|
| GET | `/commits?branch=` | `/graphs/{id}/commits?branch=` | bearer + `read` | list | `server_commit_list` |
|
|
| GET | `/commits/{commit_id}` | `/graphs/{id}/commits/{commit_id}` | bearer + `read` | show | `server_commit_show` |
|
|
|
|
Server-level management endpoints (v0.6.0+):
|
|
|
|
| Method | Path | Auth | Action | Handler |
|
|
|---|---|---|---|---|
|
|
| GET | `/graphs` | bearer + `graph_list` on `Server::"root"` | list registered graphs | `server_graphs_list` (405 in single mode) |
|
|
|
|
## Adding and removing graphs (multi mode)
|
|
|
|
Runtime add/remove via API is **not** exposed in v0.6.0 — neither
|
|
`POST /graphs` nor `DELETE /graphs/{id}` is implemented. Operators add
|
|
or remove graphs by stopping the server, editing the `graphs:` map in
|
|
`omnigraph.yaml`, then restarting. The server treats `omnigraph.yaml`
|
|
as operator-owned configuration and never writes it.
|
|
|
|
A future release may introduce a managed registry (Lance-backed,
|
|
catalog-style: reserve → init → publish with recovery sidecars) and
|
|
re-expose runtime mutation on top of it.
|
|
|
|
## Inline read queries (`POST /query`)
|
|
|
|
`POST /query` is the read-only, agent-friendly twin of `POST /read`. The
|
|
request body uses clean field names that match the CLI `-e` flag and the GQ
|
|
`query` keyword:
|
|
|
|
```json
|
|
{
|
|
"query": "query find($n: String) { match { $p: Person { name: $n } } return { $p.name } }",
|
|
"name": "find",
|
|
"params": { "n": "Alice" },
|
|
"branch": "main",
|
|
"snapshot": null
|
|
}
|
|
```
|
|
|
|
Response shape is identical to `/read` (`ReadOutput`). If the inline source
|
|
contains mutations (`insert` / `update` / `delete`), the request is rejected
|
|
with HTTP 400 and an error pointing the caller at `POST /mutate` — the
|
|
read-only contract is enforced at the URL.
|
|
|
|
`POST /mutate` is the canonical mutation endpoint. It accepts the same clean
|
|
field names (`query`, `name`); the legacy field names `query_source` and
|
|
`query_name` continue to deserialize as serde aliases so existing clients keep
|
|
working without changes.
|
|
|
|
## Deprecated names (`/read`, `/change`)
|
|
|
|
`POST /read` and `POST /change` are kept for back-compat indefinitely — they
|
|
are byte-stable on the request side and otherwise behave identically to
|
|
`/query` / `/mutate`. They are flagged as deprecated through three independent
|
|
channels:
|
|
|
|
- **OpenAPI**: the operations carry `deprecated: true` in `openapi.json`, so
|
|
every OpenAPI codegen (typescript-fetch, openapi-generator, oapi-codegen,
|
|
…) emits a `@deprecated` marker on the generated SDK method.
|
|
- **Response headers (RFC 9745)**: every response carries `Deprecation: true`.
|
|
- **Response headers (RFC 8288)**: every response carries a `Link` header
|
|
pointing at the canonical successor:
|
|
`Link: </query>; rel="successor-version"` for `/read`, and
|
|
`Link: </mutate>; rel="successor-version"` for `/change`. SDKs and HTTP
|
|
proxies can pick the successor up automatically.
|
|
|
|
Migration is purely cosmetic on the client side — swap the URL path, leave
|
|
the request body and response handling alone.
|
|
|
|
## Streaming
|
|
|
|
Only `/export` streams (`application/x-ndjson`, MPSC channel + `Body::from_stream`). Everything else is buffered JSON.
|
|
|
|
## Error model
|
|
|
|
Uniform `ErrorOutput { error, code?, merge_conflicts[], manifest_conflict? }` with `code ∈ unauthorized | forbidden | bad_request | not_found | conflict | too_many_requests | internal`. Merge conflicts attach structured `MergeConflictOutput { table_key, row_id?, kind, message }`.
|
|
|
|
`manifest_conflict` is set on **publisher CAS rejections** (HTTP 409): the
|
|
caller's pre-write view of one table's manifest version was stale.
|
|
`ManifestConflictOutput { table_key, expected, actual }` tells the client
|
|
which table to refresh and retry. This is the conflict shape produced by
|
|
concurrent `/mutate` (or its `/change` alias) or `/ingest` calls landing
|
|
the same `(table, branch)` race.
|
|
|
|
HTTP status codes used: 200, 400, 401, 403, 404, 409, 429, 500.
|
|
|
|
## Per-actor admission control
|
|
|
|
Disjoint
|
|
`(table, branch)` writes from different actors now run concurrently,
|
|
guarded only by the engine's per-(table, branch) write queue. To keep
|
|
one heavy actor from exhausting shared capacity (Lance I/O, manifest
|
|
churn, network), the server gates mutating handlers through a
|
|
`WorkloadController` configured per-process from environment variables:
|
|
|
|
| Env var | Default | Purpose |
|
|
|---|---|---|
|
|
| `OMNIGRAPH_PER_ACTOR_INFLIGHT_MAX` | 16 | Concurrent in-flight mutations per actor |
|
|
| `OMNIGRAPH_PER_ACTOR_BYTES_MAX` | 4 GiB | In-flight estimated bytes per actor |
|
|
|
|
When an actor exceeds its in-flight count or byte budget, the server
|
|
returns **HTTP 429 Too Many Requests** with `code: too_many_requests`
|
|
and a `Retry-After` header (seconds). The actor should back off; other
|
|
actors are unaffected.
|
|
|
|
Cedar policy authorization runs **before** admission accounting so
|
|
denied requests don't consume admission slots.
|
|
|
|
Today admission gates every mutating handler: `/mutate` (and its
|
|
deprecated alias `/change`), `/ingest`, `/branches/{create,delete,merge}`,
|
|
and `/schema/apply`. Read-only endpoints (`/snapshot`, `/query`, `/read`,
|
|
`/export`, `/branches` GET, `/commits`, `/schema` GET) are not
|
|
admission-gated.
|
|
|
|
## Body limits
|
|
|
|
- Default: 1 MB
|
|
- `/ingest`: 32 MB
|
|
|
|
## Auth model (`bearer + SHA-256`)
|
|
|
|
- Tokens are SHA-256 hashed on startup; plaintext is never persisted in memory.
|
|
- Constant-time comparison via `subtle::ConstantTimeEq`.
|
|
- Three sources, in precedence:
|
|
1. `OMNIGRAPH_SERVER_BEARER_TOKENS_AWS_SECRET` — AWS Secrets Manager (build with `--features aws`)
|
|
2. `OMNIGRAPH_SERVER_BEARER_TOKENS_FILE` or `OMNIGRAPH_SERVER_BEARER_TOKENS_JSON` — JSON `{actor_id: token, …}`
|
|
3. `OMNIGRAPH_SERVER_BEARER_TOKEN` — single legacy token, actor `default`
|
|
- If no tokens are configured, startup refuses unless `--unauthenticated` or
|
|
`OMNIGRAPH_UNAUTHENTICATED=1` explicitly opts into open local-dev mode. A
|
|
policy file without tokens is also rejected at startup. In open mode
|
|
`/openapi.json` strips the security scheme.
|
|
|
|
See [deployment.md](deployment.md) for token-source operational details.
|
|
|
|
## Tracing & observability
|
|
|
|
- `tower_http::TraceLayer::new_for_http()`
|
|
- Policy decisions logged at INFO level with actor, action, branch, decision, matched rule
|
|
- Startup logs: token source name, graph URI, bind address
|
|
- Graceful SIGINT shutdown
|
|
|
|
## Not implemented (by design or "TBD")
|
|
|
|
- CORS — not configured; add `tower_http::cors` if needed.
|
|
- Rate limiting — per-actor admission control gates `/mutate` (alias
|
|
`/change`), `/ingest`, `/branches/{create,delete,merge}`,
|
|
`/schema/apply` (see "Per-actor
|
|
admission control" above). No global rate limiter is configured;
|
|
add `tower_http::limit` if a graph-wide cap is needed.
|
|
- Pagination — none (commits/branches return everything; export streams).
|
|
- Runtime graph add/remove — edit `omnigraph.yaml` and restart.
|