Implement RFC-022 unified graph write protocol (#343)

* Implement unified graph write protocol

* Preserve recovery error wire compatibility

docs/user/operations/audit.md

@@ -31,13 +31,12 @@ List commits to see who made each change:
 omnigraph commit list graph.omni
 ```
 
-System-initiated writes use reserved actor ids — for example, automatic recovery
-of an interrupted write records `omnigraph:recovery`, so operator changes and
-machine repairs are distinguishable in the history:
-
-```bash
-omnigraph commit list --filter actor=omnigraph:recovery graph.omni
-```
+System-initiated writes use reserved actor ids. Rollback and legacy recovery
+commits use `omnigraph:recovery`, while a v3 roll-forward preserves the
+interrupted writer's original commit id and actor. Exact recovery actions and
+per-table outcomes are stored in the internal
+`_graph_commit_recoveries.lance` audit table; the CLI does not currently expose
+that table, so `commit list` alone is not a complete recovery log.
 
 ## What is tracked
 

docs/user/operations/errors.md

@@ -8,9 +8,11 @@
 - `Io(io::Error)`
 - `Manifest(ManifestError { kind: BadRequest|NotFound|Conflict|Internal, details: Option<ManifestConflictDetails>, … })`
   - `ManifestConflictDetails::ExpectedVersionMismatch { table_key, expected, actual }` — caller's `expected_table_versions` did not match the manifest's current latest non-tombstoned version (set by `OmniError::manifest_expected_version_mismatch`).
+  - `ManifestConflictDetails::ReadSetChanged { member, expected, actual }` — an RFC-022 prepared write's branch/head/table authority changed before physical effects. HTTP returns **409** with `read_set_conflict`. A retry must start from preparation; strict writes leave that choice to the caller.
   - `ManifestConflictDetails::RowLevelCasContention` — Lance row-level CAS rejected the publish because a concurrent writer landed the same `object_id`. Retried internally by the publisher; only surfaces if the retry budget exhausts.
   - **D₂ parse-time rejection**: a single mutation query that mixes inserts/updates with deletes errors out *before any I/O* with kind `BadRequest`. Message: `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes`. See [query-language.md](../queries/index.md) for the rule.
 - `MergeConflicts(Vec<MergeConflict>)`
+- `RecoveryRequired { operation_id, reason }` — an overlapping durable recovery intent remains unresolved. Its physical effects may already have landed, or it may still be armed before the first effect. HTTP returns **503** with `recovery_required.operation_id`. Resolve the sidecar through a read-write reopen/server restart before retrying; this is intentionally not an ordinary OCC retry.
 
 Compiler-side `CompilerError` covers parse / catalog / type / storage / plan / execution / arrow / lance / IO / manifest / unique-constraint, each with structured spans (`SourceSpan { start, end }`) for ariadne-style diagnostics. The legacy `NanoError` name remains as a deprecated compatibility alias.
 

docs/user/operations/maintenance.md

@@ -9,13 +9,13 @@
 - **Also compacts the internal `__manifest` table** (RFC-013 step 2), which accumulates one fragment per commit — it now carries the graph lineage and actor rows inline (RFC-013 Phase 7: `graph_commit` / `graph_head` rows), so on the authenticated write path every commit's actor lands here too — and otherwise makes every write's metadata scan grow with history. (The `_graph_commits.lance` / `_graph_commit_actors.lance` tables are retired, so there is no separate lineage table to compact.) It takes a simpler path than data tables: `__manifest` is read at its latest version, so compaction just advances its version in place — **no manifest publish and no recovery sidecar**. (The sidecar-free property is not because it is one commit — `compact_files` can emit a `ReserveFragments` commit before the `Rewrite`, and the auto-cleanup strip below is a further commit — but because every one of those commits is content-preserving and the table is read at its latest version, so a crash at any point leaves it readable and content-identical and the next `optimize` re-plans.) It appears in the returned stats under `table_key` `"__manifest"`. It is **not yet covered by `cleanup`**, so its version chain still grows until the cleanup half lands (it requires a cleanup-resurrection safeguard first); run `optimize` on a cadence to keep per-write metadata scans flat.
 - **`optimize` is non-destructive by construction — it never garbage-collects versions, on any table (data or internal).** Compaction rewrites fragments and advances the version; old versions stay reachable until you run `cleanup`. This holds even for a graph created by an older binary that stored an on-by-default Lance `auto_cleanup` hook: `compact_files` / `optimize_indices` commit with the hook enabled and expose no skip override, so before compacting **any** table `optimize` strips its stale `lance.auto_cleanup.*` config first, so Lance's commit-time GC hook cannot fire and silently prune `__manifest`-pinned versions. (Graphs created by current binaries store no such config; the strip is the upgrade-path safety net.) The internal-table path additionally tolerates a concurrent live writer: it runs a **bounded** rebase-and-retry, so transient contention does not fail the operator's `optimize` or the live write — but sustained contention past the retry budget surfaces a loud conflict error rather than looping forever (bounded and observable, not a silent give-up). The data-table path holds the per-table write queue while it compacts, so it does not contend with mutations on that table in the first place.
 - **Reindex (index coverage maintenance).** A scalar/FTS/vector index only covers the fragments it was built over. Rows appended after the index was built (e.g. by `load --mode merge`, whose commit does not rebuild an already-existing index) are scanned unindexed, and compaction itself rewrites fragments out of an index's coverage. `optimize` runs Lance's incremental `optimize_indices` after compaction to fold those fragments back in (a delta merge, not a full retrain), restoring full coverage so equality/range/traversal predicates stay index-accelerated. This is why a table with **no compaction work but stale index coverage still commits** a new version under `optimize`. Run `optimize` on a cadence at least as frequent as your freshness window so recently-loaded rows do not linger in the unindexed flat-scan tail.
-- **Create declared-but-missing indexes (the index reconciler).** `@index`/`@key` declares intent; `schema apply` records it but builds nothing, and `load`/`mutate` defer a column that cannot be built yet (a `Vector` column with no trainable vectors). `optimize` materializes any such declared-but-unbuilt index over the compacted layout — so it is the convergence path for an `@index` added after data exists, or a vector index whose embeddings arrived via a later `embed`. A column still not buildable (no vectors yet) is reported on the table's stat as `pending_indexes` (visible in `--json`), not treated as a failure; the next `optimize` retries. So `optimize` is the single operator-facing index reconciler: it compacts, restores coverage, **and** builds declared-but-missing indexes.
+- **Create declared-but-missing indexes (the index reconciler).** `@index`/`@key` declares intent; `schema apply`, `load`, and `mutate` build no physical indexes inline. They record or publish only their exact logical/data effects and leave all index materialization to `ensure_indices`/`optimize`. `optimize` materializes every buildable declared-but-missing index over the compacted layout — so it is the convergence path for an `@index` added after data exists, or a vector index whose embeddings arrived via a later `embed`. A column still not buildable (no vectors yet) is reported on the table's stat as `pending_indexes` (visible in `--json`), not treated as a failure; the next `optimize` retries. So `optimize` is the single operator-facing index reconciler: it compacts, restores coverage, **and** builds declared-but-missing indexes.
 - Each table's compact→reindex→publish serializes with concurrent mutations on the same table. A crash mid-operation is recovered automatically on the next open (both compaction and reindex are content-preserving, so roll-forward is always safe).
 - **Requires a recovered graph.** `optimize` refuses (errors) when a pending crash-recovery operation is present — operating on an unrecovered graph could publish a partial write that recovery would roll back. Reopen the graph to run recovery, then re-run `optimize`.
 - **Uncovered drift is skipped, not interpreted.** If a table's underlying version is ahead of the version recorded in `__manifest` and no crash-recovery record covers that movement, `optimize` reports `skipped: DriftNeedsRepair` with the manifest/head versions and leaves the table untouched. Run `omnigraph repair` to classify and explicitly publish that drift.
 - Bounded by `OMNIGRAPH_MAINTENANCE_CONCURRENCY` (default 8).
 - Returns per-table stats: `table_key, fragments_removed, fragments_added, committed, skipped, manifest_version, lance_head_version, pending_indexes` (the last lists any declared `@index` column the reconciler could not build this run, with the reason — e.g. a vector column with no trainable vectors yet).
-- **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: BlobColumnsUnsupportedByLance` (and logged) instead of compacted, and the rest of the sweep proceeds normally. **Reads and writes are unaffected** — only compaction is. Consequence: fragment count and deleted-row space on blob tables are not reclaimed; query results are never affected. A skipped blob table is also **not reindexed** in the same sweep (the skip happens before the reindex step), so its index coverage on appended rows is not refreshed by `optimize` today.
+- **Blob tables use the normal compaction and reindex path.** Lance 8.0.0+ supports blob-v2 compaction, so OmniGraph no longer has a blob-specific skip or capability gate. Fragment reclamation and index-coverage repair therefore apply to blob-bearing tables like every other table.
 
 ## `repair` — explicit
 
@@ -39,7 +39,12 @@
 - CLI guards with `--confirm`; without it, prints a preview line.
 - **Non-local consent.** Against a non-local target (an `s3://` store/cluster), `cleanup` additionally requires `--yes` on top of `--confirm`: a TTY is prompted, and a non-interactive run (no TTY, or `--json`) refuses rather than destroying. A local (`file://`) target needs only `--confirm`. The same `--yes` gate applies to overwrite `load` and `branch delete`; every maintenance run echoes its resolved target to stderr (suppress with `--quiet`).
 - **Recovery floor:** `--keep < 3` may garbage-collect versions that crash recovery needs as a rollback target. Default `--keep 10` is safe.
-- **Orphaned-branch reconciliation:** before the version GC, cleanup reclaims any per-table or commit-graph branch absent from the manifest branch list. These orphans arise when a `branch_delete` flips the manifest authority but a downstream best-effort reclaim does not complete (see [branches-commits.md](../branching/index.md)). The reconciler is idempotent (it no-ops once nothing is orphaned), runs regardless of the `keep_versions` / `older_than` values (those gate version GC only), and never reclaims `main` or system-branch forks. Reclaimed forks are logged.
+- **Requires clean recovery state.** If any durable recovery intent is pending,
+  cleanup refuses before orphan reconciliation or version GC. Reopen the graph
+  read-write (or restart the server) to resolve recovery, then rerun cleanup;
+  deleting transaction/version history while an intent is pending would make
+  exact effect ownership unverifiable.
+- **Orphaned-branch reconciliation:** before the version GC, cleanup reclaims any per-table Lance branch absent from the manifest branch list. These orphans arise when a `branch_delete` flips the manifest authority but a downstream best-effort reclaim does not complete (see [branches-commits.md](../branching/index.md)). The reconciler is idempotent (it no-ops once nothing is orphaned), runs regardless of the `keep_versions` / `older_than` values (those gate version GC only), and never reclaims `main` or system-branch forks. Reclaimed forks are logged. Graph lineage has no separate branch dataset: it lives in `__manifest`.
 
 ## Tombstones
 

docs/user/operations/server.md

@@ -162,25 +162,51 @@ Only `/export` streams (`application/x-ndjson`, MPSC channel + `Body::from_strea
 
 ## Error model
 
-Uniform `ErrorOutput { error, code?, merge_conflicts[], manifest_conflict? }` with `code ∈ unauthorized | forbidden | bad_request | not_found | conflict | too_many_requests | internal`. Merge conflicts attach structured `MergeConflictOutput { table_key, row_id?, kind, message }`.
+Uniform
+`ErrorOutput { error, code?, merge_conflicts[], manifest_conflict?, read_set_conflict?, recovery_required? }`
+with
+`code ∈ unauthorized | forbidden | bad_request | not_found | method_not_allowed | conflict | too_many_requests | internal`.
+Merge conflicts attach structured
+`MergeConflictOutput { table_key, row_id?, kind, message }`.
 
-`manifest_conflict` is set on **concurrent-write rejections** (HTTP 409): the
-caller's pre-write view of one table's manifest version was stale.
-`ManifestConflictOutput { table_key, expected, actual }` tells the client
-which table to refresh and retry. This is the conflict shape produced by
-concurrent `/mutate` (or its `/change` alias), `/load` (or its deprecated
-`/ingest` alias) calls landing the same `(table, branch)` race.
+`manifest_conflict` is set on legacy per-table manifest-version rejections
+(HTTP 409). `ManifestConflictOutput { table_key, expected, actual }` tells the
+client which table was stale. Mutation and load use the unified coarse-OCC
+adapter described next; other writers retain this older conflict shape until
+they are enrolled.
 
-HTTP status codes used: 200, 400, 401, 403, 404, 409, 429, 500.
+`read_set_conflict` is set when a prepared write is rejected before any table
+effect because its branch authority changed. The HTTP status is 409 and
+`ReadSetConflictOutput { member, expected, actual }` identifies the stale
+authority member. The engine already performs a bounded full-attempt retry for
+mutation inserts and load `append`/`merge`. Strict mutation updates/deletes and
+load `overwrite` return the 409 to the caller instead of being replayed.
+
+`recovery_required` is set when an overlapping durable recovery intent remains
+unresolved; its table effects may or may not have started. The HTTP status is 503 and
+`RecoveryRequiredOutput { operation_id }` names the durable recovery intent.
+The optional `code` field is omitted for this response: adding a new value to
+the closed error-code enum would break older clients, while the optional
+structured field is additive and rolling-safe.
+Do not blindly resubmit the write: let a read-write open or the recovery sweep
+resolve that operation first, then retry from a fresh snapshot.
+
+HTTP status codes used: 200, 400, 401, 403, 404, 405, 409, 429, 500, 503.
 
 ## Per-actor admission control
 
-Disjoint
-`(table, branch)` writes from different actors now run concurrently,
-guarded only by the engine's per-(table, branch) write queue. To keep
-one heavy actor from exhausting shared capacity (Lance I/O, manifest
-churn, network), the server gates mutating handlers through per-process
-admission limits configured from environment variables:
+RFC-022-enrolled mutation/load preparation runs outside the effect gates, so
+parsing, validation, and reclaimable fragment staging can overlap across branches.
+Readers acquire none of these gates. Before the first durable effect, however, an
+attempt acquires the exclusive root schema gate, then its branch-effect gate and
+sorted table queues, and holds all of them through manifest publication. The root
+schema gate means enrolled effect windows on one graph currently serialize
+in-process even across different branches; the branch gate preserves one atomic
+graph-head validation authority, while table queues protect each concrete Lance
+effect and legacy writer. These are process-local ordering gates, not a
+cross-process lock. To keep one heavy actor from exhausting shared capacity
+(Lance I/O, manifest churn, network), the server gates mutating handlers through
+per-process admission limits configured from environment variables:
 
 | Env var | Default | Purpose |
 |---|---|---|