apunkt/omnigraph
1
0
Fork 0
mirror of https://github.com/ModernRelay/omnigraph.git synced 2026-06-21 02:28:07 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

mr-668: remove POST /graphs and CLI graphs create (defer runtime graph mgmt)

Browse source 
The POST /graphs runtime-create endpoint shipped in PR 7/10 has three
unresolved high-severity bugs:

  - flock-on-renamed-inode race: the YAML flock is taken on
    omnigraph.yaml itself, then a temp file is renamed over it.
    Cross-process writers end up locking different inodes — both
    believing they hold exclusive access.
  - duplicate-check outside the file lock: precheck runs against
    the in-memory registry only; the locked closure does
    config.graphs.insert(...) unconditionally. Concurrent same-id
    POSTs can persist the loser in YAML while the in-memory registry
    keeps the winner — they disagree after restart.
  - best_effort_cleanup_init_artifacts deletes _schema.pg /
    _schema.ir.json / __schema_state.json on any init failure. An
    accidental re-init against an existing graph's URI destroys its
    schema; subsequent open() fails at read_text(_schema.pg).

The correct fix is a Lance-style cluster catalog (reserve → init →
publish with recovery sidecars), parallel to the engine's existing
__manifest discipline. That work is out of scope for v0.7.0.

For now, disable runtime add/remove from the network and CLI surface.
Operators add graphs by editing omnigraph.yaml and restarting. The
GET /graphs read-only enumeration stays.

Removed:
- POST /graphs handler + router fragment + utoipa registration
- 13 post_graphs_* server tests + 3 composite POST tests +
  multi_mode_app_with_real_config / post_graph helpers
- CLI omnigraph graphs create subcommand + its handler + cli.rs tests
- system_remote.rs combined list+create test trimmed to list-only
- YAML rewrite infra: rewrite_atomic[_with_modify], RewriteAtomicError,
  staging_path, hash_config_file, AppState::config_hash field +
  threading through new_multi and open_multi_graph_state
- fs2 dependency (verified absent from cargo tree)
- sha2/fs2 imports in config.rs (only the rewrite path used them)
- Cedar PolicyAction::GraphCreate variant + "graph_create" match arms
  + action def in Cedar schema + graph_create_action_authorizes_against_server_resource test
- GraphCreateRequest / GraphCreateResponse / GraphSchemaSpec /
  GraphPolicySpec API types (only the POST handler / CLI imported them)

Kept:
- GET /graphs (read-only enumeration) and graph_list Cedar action
- omnigraph graphs list CLI subcommand
- All multi-graph startup, mode inference, cluster routes,
  per-graph + server-level Cedar policies
- server_settings_drive_multi_graph_startup_end_to_end (the test
  that covers operator-authored YAML + restart — the path that
  survives)
- best_effort_cleanup_init_artifacts and the three init failpoints
  (still reachable from CLI `omnigraph init`; preflight fix deferred
  as a follow-up)
- GraphRegistry::insert and its concurrency tests — production
  callers gone, but the method is the natural seam for the future
  cluster-catalog work

Also fixed (transcript issue 4):
- ALWAYS_FLAT_PATHS now includes /graphs so multi-mode OpenAPI
  advertises the management route correctly (was previously rewritten
  to /graphs/{graph_id}/graphs)
- multi_mode_openapi_keeps_healthz_flat → renamed to
  multi_mode_openapi_keeps_management_paths_flat, asserts both
  /healthz and /graphs stay flat
- multi_mode_openapi_prefixes_operation_ids_with_cluster skips
  /graphs in addition to /healthz

Doc fixes:
- docs/user/cli.md: graphs list example was --target http://...,
  but --target is a config-graph-name lookup; corrected to --uri.
  Removed the graphs create example.
- docs/user/server.md: dropped POST /graphs row, "omnigraph.yaml
  ownership", and "POST /graphs body shape" sections. Added a
  paragraph stating runtime add/remove is not exposed in v0.7.0.
- docs/user/policy.md: dropped graph_create action; reworded the
  "Configuration" line to clarify that server-scoped rules (graph_list)
  take neither branch_scope nor target_branch_scope.
- docs/releases/v0.7.0.md: rewrote release narrative — multi-graph
  mode ships; runtime add/remove deferred.
- AGENTS.md: HTTP server bullet and capability matrix row updated to
  reflect read-only GET /graphs and the operator-edit workflow.
- openapi.json regenerated; /graphs has only .get, no .post.

Diff: 17 files, +123 −1525 LOC.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ragnor Comerford 2026-05-26 17:49:38 +02:00
parent d11c18fb27
commit 937fd6382d
No known key found for this signature in database
18 changed files with 136 additions and 1727 deletions
Download patch file Download diff file Expand all files Collapse all files

192
openapi.json
View file

@ -640,121 +640,6 @@
"bearer_token": []
}
]
},
"post": {
"tags": [
"management"
],
"summary": "Create a new graph at runtime (MR-668 PR 7).",
"description": "Multi-graph mode only. Operators add a graph to the registry\nwithout restarting the server. The server `Omnigraph::init`s the\nnew graph at `req.uri`, atomically rewrites `omnigraph.yaml` to\ninclude the new entry, then publishes the handle in the registry.\n\nCedar-gated by `PolicyAction::GraphCreate` against\n`Omnigraph::Server::\"root\"` (the same server-level policy as\n`GET /graphs`).\n\nFailure modes:\n* Init fails → orphan storage files at `req.uri` (PR 2a cleans up\n schema files but not Lance datasets; operator removes manually).\n* Rewrite fails (`fs2::flock` IO error) → orphan storage; YAML\n unchanged.\n* YAML drift (operator edited the file) → 503; YAML and storage\n both unchanged.\n* Duplicate `graph_id` or `uri` → 409; storage already in use.",
"operationId": "createGraph",
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/GraphCreateRequest"
}
}
},
"required": true
},
"responses": {
"201": {
"description": "Graph created",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/GraphCreateResponse"
}
}
}
},
"400": {
"description": "Invalid request body (graph_id, schema, policy file)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorOutput"
}
}
}
},
"401": {
"description": "Unauthorized",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorOutput"
}
}
}
},
"403": {
"description": "Forbidden",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorOutput"
}
}
}
},
"405": {
"description": "Method not allowed (single-graph mode)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorOutput"
}
}
}
},
"409": {
"description": "graph_id or uri already registered",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorOutput"
}
}
}
},
"413": {
"description": "Request body too large (>32 MiB)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorOutput"
}
}
}
},
"500": {
"description": "Init failure or YAML rewrite failure",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorOutput"
}
}
}
},
"503": {
"description": "omnigraph.yaml drift detected (operator edited the file)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorOutput"
}
}
}
}
},
"security": [
{
"bearer_token": []
}
]
}
},
"/healthz": {
@ -1440,56 +1325,6 @@
}
}
},
"GraphCreateRequest": {
"type": "object",
"description": "Request body for `POST /graphs` (MR-668 PR 7).\n\nBody shape:\n```json\n{\n \"graph_id\": \"alpha\",\n \"uri\": \"/path/to/alpha.omni\",\n \"schema\": { \"source\": \"<inline .pg source>\" },\n \"policy\": { \"file\": \"./policies/alpha.yaml\" }\n}\n```\n\n32 MiB body limit (matches `INGEST_REQUEST_BODY_LIMIT_BYTES`).",
"required": [
"graph_id",
"uri",
"schema"
],
"properties": {
"graph_id": {
"type": "string",
"description": "New graph's id. Must satisfy `^[a-zA-Z0-9-]{1,64}$`, not start with\n`_`, and not be a reserved name. See `GraphId::try_from`."
},
"policy": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/GraphPolicySpec",
"description": "Per-graph Cedar policy. Optional — `None` means the graph has\nno per-graph policy enforcement (HTTP auth still applies if\nconfigured)."
}
]
},
"schema": {
"$ref": "#/components/schemas/GraphSchemaSpec",
"description": "Inline schema (`{ source }`). Required."
},
"uri": {
"type": "string",
"description": "Storage URI (local path or `s3://...`). Must NOT already be in\nuse by another registered graph. Server `Omnigraph::init`s the\ngraph at this URI."
}
}
},
"GraphCreateResponse": {
"type": "object",
"description": "Response from `POST /graphs` on success (201 Created).",
"required": [
"graph_id",
"uri"
],
"properties": {
"graph_id": {
"type": "string"
},
"uri": {
"type": "string"
}
}
},
"GraphInfo": {
"type": "object",
"description": "One entry in the response from `GET /graphs`. Cluster operators\nconsume this list to discover which graphs the server is currently\nserving. The shape is intentionally minimal — `graph_id` and `uri`\nare the only fields a routing client needs.",
@ -1521,33 +1356,6 @@
}
}
},
"GraphPolicySpec": {
"type": "object",
"description": "Per-graph policy specification in `POST /graphs`. Mirrors the\n`policy: { file }` shape in `omnigraph.yaml`'s `graphs.<id>.policy`\nsection.",
"properties": {
"file": {
"type": [
"string",
"null"
],
"description": "Path to the per-graph Cedar policy file, server-side.\nMust be readable by the server process at request time.\nPath is relative to the server's working directory (NOT to the\n`omnigraph.yaml`'s `base_dir`) — caller-supplied paths are\ntrusted as-is."
}
}
},
"GraphSchemaSpec": {
"type": "object",
"description": "Schema specification for a new graph in `POST /graphs`. Nested\nper MR-668 decision 7 — leaves room for future fields without\nbreaking the request shape. Mirrors the `policy: { file }` nesting\npattern.\n\nToday only `source` (inline `.pg` text) is supported. Future fields\nmight include `schema.allow_data_loss`, `schema.version`, etc.\n\n**Asymmetric with `SchemaApplyRequest`**: `POST /schema/apply` still\nuses a flat `schema_source: String` for backwards compatibility.\nA follow-up release may migrate that too.",
"required": [
"source"
],
"properties": {
"source": {
"type": "string",
"description": "Inline `.pg` schema source.",
"example": "node Person {\n name: String @key\n age: I32?\n}"
}
}
},
"HealthOutput": {
"type": "object",
"required": [