Harden branch recovery and cleanup retention (#344)

* Harden branch recovery and cleanup retention * Fail closed on blocked partial rollback
2026-07-12 03:12:11 +02:00 · 2026-07-11 16:56:40 +03:00 · 2026-07-11 16:56:40 +03:00 · bd4c614e42
commit bd4c614e42
parent f758ff0d17
25 changed files with 2227 additions and 213 deletions
--- a/docs/dev/invariants.md
+++ b/docs/dev/invariants.md
@ -159,7 +159,8 @@ converge the physical state.
 | Multi-table commit | Manifest CAS plus recovery sidecars; not a single Lance primitive | [writes.md](writes.md), [architecture.md](architecture.md) |
 | Constructive mutations | In-memory `MutationStaging`, one end-of-query table commit per touched table, then one manifest publish | [writes.md](writes.md), [execution.md](execution.md) |
 | Deletes | Staged like inserts/updates (`stage_delete` via Lance 7.0 `DeleteBuilder::execute_uncommitted`, MR-A) — no inline HEAD advance; mixed insert/update/delete in one query rejected by D2 as a deliberate boundary (constructive XOR destructive per query; compose via separate mutations or a branch) | [query-language.md](../user/queries/index.md), [writes.md](writes.md) |
-| Branch delete | Manifest is the single authority, flipped atomically first; per-table forks are derived state, reclaimed best-effort (`force_delete_branch`) with the `cleanup` reconciler as the guaranteed backstop. Under schema/target/all-table gates, a target-scoped unresolved sidecar may be made unreachable by deletion and is then audit-discarded by recovery; graph-global SchemaApply still blocks. Reusing a name whose fork reclaim failed before `cleanup` surfaces an actionable error | [branches-commits.md](../user/branching/index.md), [maintenance.md](../user/operations/maintenance.md), [writes.md](writes.md) |
+| Branch create/delete | `__manifest` `BranchContents` is the single logical authority. Lance create is physically two-phase, so OmniGraph prevalidates names, enforces path-prefix-disjoint live graph names, reclaims an absent-ref clone-only tree, and uses a bounded completion classifier; delete removes authority before tree cleanup, so an absent ref is success and derived tree reclaim may converge later. Neither control emits graph lineage. Per-table forks are derived state, reclaimed best-effort with `cleanup` as backstop. Under schema/target/all-table gates, a target-scoped unresolved sidecar may be made unreachable by deletion and is then audit-discarded by recovery; graph-global SchemaApply still blocks | [branches-commits.md](../user/branching/index.md), [maintenance.md](../user/operations/maintenance.md), [writes.md](writes.md) |
+| Cleanup retention | Explicit cleanup derives exact `keep` cutoffs from Lance's available version list, caps each main-table GC cutoff at the oldest exact main version inherited by any live lazy graph branch, and refuses uncovered main HEAD drift. Lance protects native per-table branch refs itself. The graph-wide live-reference preflight fails closed before the first table GC, after which individual table failures remain fault-isolated | [maintenance.md](../user/operations/maintenance.md), [writes.md](writes.md) |
 | Schema validation | Type checks, required fields, defaults, edge endpoint checks, and edge cardinality are enforced on write paths | [schema-language.md](../user/schema/index.md), [execution.md](execution.md) |
 | Unique constraints | Value/enum, uniqueness, edge-RI, and cardinality route through ONE unified, catalog-derived evaluator (`crate::validate`) on ALL THREE write surfaces — branch-merge, mutation, and bulk load: Δ-scoped (checks the delta, not the whole graph) and structured-filter-backed (committed probes use a BTREE when reconciled and remain correct by scanning while it is pending), reusing the leaf checks (`loader::validate_value_constraints`/`validate_enum_constraints`/`composite_unique_key`) so the surfaces cannot drift. This closed the prior merge bug (merge validated `@range`/`@check` but not enum) AND the **cross-version uniqueness gap** on the mutation and load paths (a duplicate of a committed `@unique` value is now rejected; the merge path always enforced it). The committed view is the merge target snapshot (merge), the write's pinned `txn.base` (mutation), or the pinned pre-load base (load — `Overwrite` validates the batch as the whole new image, committed view empty); `@card` refreshes the manifest-visible graph-branch snapshot on the mutation path only (the #298 stale-handle fix), then follows each entry's actual inherited/owned Lance ref. `@key` is id-backed, so it is checked intra-delta only (a committed holder of a key value is always the same row — an upsert), skipping a wasted O(Δ) probe per keyed row; `@unique` (non-key) groups do the committed lookup. | [schema-language.md](../user/schema/index.md) |
 | Storage trait | `TableStorage` (via `db.storage()`) is staged-only; the sole inline-commit residual (`create_vector_index`) is split onto a separate sealed `InlineCommitResidual` trait reached via `db.storage_inline_residual()` (MR-854), so §1 holds by construction; capability/stat surfaces are roadmap | [writes.md](writes.md), [architecture.md](architecture.md) |
@ -168,12 +169,23 @@ converge the physical state.
 | Auth | Bearer token hashing and server-side actor resolution are implemented at the HTTP boundary | [server.md](../user/operations/server.md), [policy.md](../user/operations/policy.md) |
 | Tests | Tempdir-backed Lance tests are the current substrate; the storage adapter has an in-memory backend for adapter-level contract tests, but Lance datasets bypass it | [testing.md](testing.md) |

-The branch-delete reconciler is authority-derived: it reclaims orphaned forks
-today and degrades to a no-op if Lance ships an atomic multi-dataset branch
-operation, so the design composes with that future rather than blocking it. This
-is the same shape as invariant 7 (indexes are derived state); prefer it over a
-recovery-sidecar-style approach for any new multi-dataset metadata operation,
-since the sidecar would be scaffolding to remove once the substrate closes the gap.
+The branch-control reconcilers are authority-derived: same-name create reclaims
+an absent-ref clone-only manifest tree, and delete/cleanup reclaim orphaned
+per-table forks. They degrade to no-ops if Lance closes the corresponding
+physical gaps, so the design composes with that future rather than blocking it.
+This is the same shape as invariant 7 (indexes are derived state); prefer it
+over a recovery-sidecar-style approach for metadata work whose logical target
+is fully derivable from one existing authority. A graph-visible table effect is
+different and still requires durable ownership before it can advance HEAD.
+For legacy path-prefix overlaps, an ancestor first-touch tree is not proven
+unreachable while a live child remains. Full recovery may leave the sidecar
+intact and allow the graph to open for leaf-first child deletion **only when the
+intent owns no physical table effect**. If the same sidecar owns any durable
+effect, cleanup is part of an all-or-nothing rollback: read-write open fails
+closed until the child is removed through an existing handle or an offline
+Lance-level branch tool, then the next Full sweep reclaims the untouched fork and
+compensates the owned effect. Returning a writable handle with that rollback
+incomplete would let a legacy writer prepare from unresolved physical state.

 ## Known Gaps

@ -252,19 +264,24 @@ them explicit.
  files and commits. Both `reclaim_orphaned_fork_and_refork` and
  `reconcile_orphaned_branches` consult pending sidecars before destruction; a
  foreign claim is conflict/indeterminate, never permission to delete. Full
-  recovery accepts sidecar-before-ref crashes and deletes an unpublished fork
-  only when it is still exactly at the inherited version and no other sidecar
-  claims `(table_path, target ref)`. A no-effect sidecar with a competitor
+  recovery accepts sidecar-before-ref crashes. When `BranchContents` is absent,
+  the already-armed intent plus the single-writer-process gate makes a same-name
+  clone-only tree unreachable, so recovery force-reclaims it without pretending
+  it can inspect an unlisted version. When a ref exists, recovery deletes the
+  unpublished fork only after proving it is still exactly at the inherited
+  version and no other sidecar claims `(table_path, target ref)`. A no-effect
+  sidecar with a competitor
  discards only itself; the last survivor either cleans the untouched ref or
  recovers its owned effect. Partial rollback performs no-effect cleanup before
  publishing its fixed outcome. This closes the cross-handle live-ref
  deletion bug and keeps cleanup as the backstop for truly unclaimed refs.
-  The remaining multi-process gap is narrower but real: Lance exposes
-  `force_delete_branch`, not compare-and-delete by `BranchIdentifier`, so a
-  foreign process can create intent/ref between the final list/check and the
-  delete. The documented single-writer-process support boundary remains until
-  Lance provides a conditional ref primitive (or OmniGraph adds a distributed
-  fence); process-local queues are not credited as that primitive.
+  The remaining multi-process gap is narrower but real: Lance exposes neither
+  conditional native graph-branch create/delete nor compare-and-delete by
+  `BranchIdentifier` for per-table refs, so a foreign process can mutate a ref
+  between the final list/check and the operation. The documented
+  single-writer-process support boundary remains until Lance provides a
+  conditional ref primitive (or OmniGraph adds a distributed fence);
+  process-local queues are not credited as that primitive.
 - **Local `write_text_if_match` is not a cross-process CAS:** object-store
  backends use a true conditional put (ETag If-Match; the in-memory test
  backend too), but upstream `object_store` leaves `PutMode::Update`
--- a/docs/dev/lance.md
+++ b/docs/dev/lance.md
@ -198,6 +198,20 @@ findings:
  the substrate enforcing what omnigraph's entry-owner resolution always did.
  Production opens by owner-resolved location (unaffected); the lazy-fork
  namespace test pins both the error and the owner-branch open.
+- **Native branch control remains two-phase at beta.15.**
+  `Dataset::create_branch` commits the shallow-cloned `tree/{branch}` dataset
+  before validating/writing authoritative `BranchContents`; its random
+  `BranchIdentifier` element is minted only in that second phase and cannot be
+  pre-minted through the public API. OmniGraph therefore validates first and
+  classifies an ambiguous result from exact parent metadata rather than
+  inventing an identifier. Delete removes `BranchContents` before tree cleanup
+  and exposes no compare-and-delete primitive. Slash-separated names share
+  nested physical directories; `force_delete_branch` deliberately leaves an
+  ancestor tree while a live path-child exists, so live graph names are
+  path-prefix-disjoint. The public format page does not currently spell out the
+  identifier field, so the pinned Rust shape remains load-bearing. Guard 9 in
+  `lance_surface_guards.rs` pins clone-only raw-create failure plus force reclaim;
+  `src/branch_control.rs` pins delete classification and JSON identity fencing.
 - **Native DirectoryNamespace churn** (#7222 removed
  `table_version_storage_enabled` + the `__manifest` version-storage
  experiment; #7176/#7191/#7234 rewrote manifest handling): the decoupling
@ -287,7 +301,7 @@ Migration from Lance 4.0.0 → 6.0.1 landed in this cycle (DataFusion 52 → 53,
 - **Lance #6658 closed** (2026-05-14) but `DeleteBuilder::execute_uncommitted` did **not** ship in v6.0.1 — binary search across the release stream shows it first appears in `v7.0.0-beta.10` (the closing commits landed on main but didn't backport to the 6.x line). Tracked as MR-A: migrate `delete_where` to staged, retire the parse-time D2 mutation rule, extend recovery sidecar coverage. **Gated on the Lance v7.x bump**, not this PR. v7.0.0-rc.1 dropped 2026-05-21.
 - **Lance #6666 still open** (`build_index_metadata_from_segments` public): vector-index two-phase blocked; inline `create_vector_index` residual retained.
 - **Lance #6877 still open** (`MergeInsertBuilder` dup-rowid): PR #109's `SourceDedupeBehavior::FirstSeen` + `check_batch_unique_by_keys` precondition stay load-bearing.
- **`Dataset::force_delete_branch`** (`branches().delete(name, force=true)`, dataset.rs:524) tolerates a missing branch-*contents* ref (vs plain `delete_branch`'s `RefNotFound`), but on the local store still errors `NotFound` if the branch `tree/` directory is fully absent (`remove_dir_all`'s NotFound is not caught for Lance's native error variant, refs.rs:526-549). Both variants still refuse a branch with referencing descendants (`RefConflict`). `TableStore::force_delete_branch` wraps this to be fully idempotent (tolerates already-absent). The single-authority branch-delete redesign uses it for orphan reclamation (eager best-effort reclaim + cleanup reconciler). Pinned by `lance_surface_guards.rs::force_delete_branch_semantics`. Branch delete is "flip the ref atomically, then `remove_dir_all(tree/{branch})`"; branch-exclusive data lives under `tree/{branch}/` so a drop reclaims it immediately without touching `main`.
+- **`Dataset::force_delete_branch`** (`branches().delete(name, force=true)`, dataset.rs:524) tolerates a missing branch-*contents* ref (vs plain `delete_branch`'s `RefNotFound`), but on the local store still errors `NotFound` if the branch `tree/` directory is fully absent (`remove_dir_all`'s NotFound is not caught for Lance's native error variant, refs.rs:526-549). Both delete variants still refuse a branch with referencing descendants (`RefConflict`). The current OmniGraph disposition for these still-present substrate behaviors is recorded in the beta.15 audit above.
 - **Lance blob-v2 `compact_files` bug** (no public issue found as of 2026-06): `compact_files` disables binary-copy for blob datasets and forces `BlobHandling::AllBinary` on the read side; the v2.1+ structural decoder then mis-counts column infos for the blob-v2 struct and fails with `Invalid user input: there were more fields in the schema than provided column indices / infos` (`lance-encoding/src/decoder.rs::ColumnInfoIter::expect_next`). This fails even a pristine uniform-V2_2 multi-fragment blob table; vector/list/scalar/ragged columns and mixed file versions all compact fine. Reads/queries use descriptor handling (`BlobHandling::default()`) and are unaffected. `optimize` skips blob-bearing tables behind `LANCE_SUPPORTS_BLOB_COMPACTION = false` (`db/omnigraph/optimize.rs`), reporting `SkipReason::BlobColumnsUnsupportedByLance`. Pinned by `lance_surface_guards.rs::compact_files_still_fails_on_blob_columns`, which turns red when the bug is fixed → flip the gate, remove the skip branch + the `maintenance.rs::optimize_skips_blob_table_and_reports_skip` skip assertions.

 Surface guards added: `crates/omnigraph/tests/lance_surface_guards.rs` (10 named guards; 5 runtime + 5 compile-only; plus the index-coverage work's `_compile_optimize_indices_signature` and `optimize_indices_extends_fragment_coverage`). Future Lance bumps re-run this file first as the smoke check. Two additional guards from the original plan deferred to follow-up (`manifest_cas_returns_row_level_contention_variant` needs full publisher-race harness; `table_version_metadata_byte_compatible_with_v4` needs `pub(crate)` reach extension).
--- a/docs/dev/rfc-022-027-architecture-review.md
+++ b/docs/dev/rfc-022-027-architecture-review.md
@ -37,17 +37,19 @@ The central architecture remains the right one:
  separate irreversible decisions and should keep separate evidence and format
  gates.

-The current blockers are boundary problems, not a reason to replace that core.
-They concern crash classification, actual fencing, foreign-branch authority,
-and capability activation.
+The remaining open blockers are boundary problems, not a reason to replace that
+core. They concern actual cross-process fencing, capability activation, format
+rollout, and public RFC lifecycle. The native branch crash classifier,
+foreign-source merge semantics, and shipped lazy-branch cleanup exposure found
+by this review are now dispositioned below.

 > 💬 **Second-pass verification (2026-07-11):** I independently re-verified this
 > ledger's two sharpest new substrate claims against the pinned checkout before
 > commenting: BLOCKER-01's two-phase branch create is confirmed in Lance's own
 > doc comment, and BLOCKER-04's lazy-branch exposure is confirmed
-> architecturally — and appears to be a live bug in the shipped `cleanup`, not
-> only an RFC gap (see the comment there). Overall judgment: concur with this
-> ledger; three of its findings supersede corrections I applied to the RFCs on
+> architecturally. Both findings produced shipped fixes on 2026-07-11 and their
+> durable dispositions are recorded below. Overall judgment: concur with this
+> ledger; three other findings supersede corrections applied to the RFCs on
 > 2026-07-11 (noted inline at BLOCKER-07, BLOCKER-11, and tightening 5).

 The latest revisions improved several important points and those changes should
@ -98,7 +100,9 @@ sub-contract of an unaccepted RFC as a separately accepted decision.

 **Affected:** [RFC-022 §7](../rfcs/rfc-022-unified-write-path.md#7-native-graph-branch-ref-control-protocol)

-RFC-022 currently models branch create/delete as one native ref mutation whose
+**Status:** Closed in specification and implementation on 2026-07-11.
+
+At review time RFC-022 modeled branch create/delete as one native ref mutation whose
 completion is the visibility point. The pinned Lance implementation is more
 specific:

@ -143,6 +147,21 @@ and between delete authority removal and directory cleanup.
 > upstream ask) addressed the *conditional-put* gap but not this crash
 > classification — the two dispositions compose; both are needed.

+> ✅ **Disposition (2026-07-11):** RFC-022 §7 now makes `BranchContents` the sole
+> logical authority, defines the bounded create/delete classifier, explicitly
+> rejects a graph-branch sidecar/lineage entry, and retains the
+> single-writer-process boundary. The implementation prevalidates names,
+> requires live graph names to be physical-path-prefix-disjoint, reclaims an
+> absent-ref clone before one bounded retry, accepts only a current-attempt
+> matching lost acknowledgement, and fences delete by exact identifier. Full
+> first-touch recovery also force-reclaims clone-only table residue under its
+> already-durable writer sidecar. Coverage includes flat and named-source
+> clone-only recovery, invalid-name-before-clone, lost acknowledgements,
+> absent-ref/tree-present delete, same-identifier error preservation,
+> recreated-identifier refusal, path-prefix collision, legacy no-effect
+> leaf-first delete, mixed-effect rollback fail-closed behavior, and no
+> manifest-version/lineage movement.
+
 ### BLOCKER-02 — create-if-absent ownership is not a fencing lease

 **Affected:** RFC-023 migration claim, RFC-024 migration claim, and
@ -193,6 +212,9 @@ that takeover is refused until external fencing/death proof is established.
 [RFC-023 §6](../rfcs/rfc-023-key-conflict-fencing.md#6-retry-and-validation-semantics),
 and [RFC-026 §6](../rfcs/rfc-026-memwal-streaming-ingest.md#6-fold-protocol)

+**Status:** The RFC-022 mutation/load portion is closed in the shipped adapter
+as of 2026-07-11. RFC-023 and RFC-026 retain their own open dispositions.
+
 A retryable concurrent inserted-row-filter conflict is detected when a table
 transaction attempts to commit. (`WhenMatched::Fail` may instead report a
 pre-existing match during merge execution.) In a multi-table graph write or
@ -217,10 +239,24 @@ attempt around that sidecar. For non-strict upsert, the barrier must resolve the
 sidecar before the bounded automatic whole-operation retry. Add a failpoint/race
 in which table N conflicts after table 1 has committed.

+> ✅ **RFC-022 disposition (2026-07-11):** mutation/load take the conservative
+> safe branch of this requirement. A pre-effect authority mismatch discards and
+> fully reprepares the attempt. Once the exact-effect sidecar is durable and the
+> commit phase begins, every Lance commit failure returns `RecoveryRequired` and
+> leaves that sidecar intact, even when Full recovery later proves that no table
+> effect landed. The adapter performs zero transparent Lance commit retries and
+> never prepares a new semantic attempt around unresolved ownership; the next
+> synchronous barrier classifies and resolves it first. Finalizing a proven-empty
+> intent and automatically retrying would improve ergonomics, but is not required
+> for safety. RFC-023's fenced key-conflict contract and RFC-026's MemWAL fold
+> protocol still need their adapter-specific resolutions.
+
 ### BLOCKER-04 — live graph branches need physical GC protection

 **Affected:** [RFC-025 §6](../rfcs/rfc-025-checkpoint-retention.md#6-cleanup-protocol-and-pruned-through-boundary)

+**Status:** Closed in the shipped baseline and RFC-025 on 2026-07-11.
+
 The cleanup root set includes live graph branches, but RFC-025 creates Lance
 tags only for named checkpoints. This is insufficient for lazy graph branches.
 A graph branch may store, in its `__manifest`, a foreign reference to an old
@ -253,6 +289,18 @@ runs, and the lazy branch still opens and reads the exact pinned state.
 > mechanism but are history-trimming under an operator-confirmed policy; a
 > live branch's working state is not history.)

+> ✅ **Disposition (2026-07-11):** shipped cleanup now resolves main plus every
+> live non-system graph branch under schema → all-branch → all-table gates,
+> verifies every exact inherited-main version opens, and caps each dataset's
+> cutoff at the oldest such pin before the first table GC. It also derives
+> `keep=N` from Lance's actual ordered version list and refuses uncovered main
+> HEAD drift. Unclassifiable roots abort graph-wide; only later per-table GC
+> failures are fault-isolated. Regression coverage pins count- and time-based
+> retention, the oldest of multiple live pins, exact keep counts, graph-wide
+> fail-closed ordering, and uncovered drift. RFC-025 §6 now composes checkpoint
+> tags with this required lazy-branch floor rather than treating tags as a
+> replacement.
+
 ### BLOCKER-05 — durable-head OCC must compare the full head token

 **Affected:** [RFC-024 §5](../rfcs/rfc-024-durable-table-heads.md#5-atomic-publisher-contract)
@ -382,6 +430,11 @@ retaining the public lifecycle status `Proposed`.

 ### BLOCKER-10 — do not put foreign-branch facts in an atomic `ReadSet`

+**Affected:** [RFC-022 §6.2](../rfcs/rfc-022-unified-write-path.md#62-branch-merge)
+
+**Status:** Closed in specification on 2026-07-11; full merge-adapter conversion
+remains rollout work rather than an architecture ambiguity.
+
 RFC-022 requires every `ReadSet` member to be arbitrated atomically by the
 publish CAS. A CAS on reserved main cannot arbitrate a row on a named source
 branch; a merge-target CAS cannot arbitrate a source-branch row.
@ -400,6 +453,13 @@ Use the right category for each fact:

 Keep target-branch values that must remain stable in the atomic `ReadSet`.

+> ✅ **Disposition (2026-07-11):** RFC-022 §6.2 defines the exact captured source
+> commit/snapshot as an immutable effect precondition, not a member of the
+> target publisher's atomic `ReadSet`. A later source-head advance is harmless
+> under captured-source semantics; only a future latest-at-target-publish
+> contract would require a source fence through target CAS. The current bridge
+> remains conservatively stricter before effects while its full adapter lands.
+
 ### BLOCKER-11 — RFC-022 and no-heads MemWAL need coarse OCC

 **Affected:** [RFC-022 §10](../rfcs/rfc-022-unified-write-path.md#10-rollout-and-compatibility)
@ -525,7 +585,9 @@ RFC set is merged:
 The review does not require all RFCs to land together. A safe order is:

 1. accept RFC-022 after branch-control recovery, coarse `graph_head` OCC, and
-   foreign-branch fact classification are explicit;
+   foreign-branch fact classification are explicit (all three are now
+   dispositioned; the public lifecycle decision in BLOCKER-09 still governs
+   what “accepted” means);
 2. accept and land the stable identity RFC/capability;
 3. accept RFC-023 once partial-effect retry and its format/fleet barrier are
   complete;
@ -540,10 +602,8 @@ The review does not require all RFCs to land together. A safe order is:
 This ordering preserves the split's main benefit: a blocked performance
 optimization or research result does not hold correctness work hostage.

-> 💬 **Concur with the order; two sequencing notes (2026-07-11):**
-> BLOCKER-04's escalation means step 5 has a prerequisite outside this list —
-> the live lazy-branch cleanup exposure needs its red regression test and fix
-> in the shipped `cleanup` first, so RFC-025 builds on a correct baseline. And
-> step 3's TIGHTENING-01 demotion hinges on the partial-update filter check
+> 💬 **Order update (2026-07-11):** BLOCKER-04's shipped-cleanup prerequisite is
+> now complete and RFC-025 §6 incorporates its floor. Step 3's TIGHTENING-01
+> demotion still hinges on the partial-update filter check
 > noted there; if that check lands `None`, upstream symmetry returns to the
 > activation gate and RFC-023's timeline moves accordingly.
--- a/docs/dev/testing.md
+++ b/docs/dev/testing.md
@ -19,14 +19,14 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav
 | File | Covers |
 |---|---|
 | `end_to_end.rs` | Full init → load → query/mutate flow |
-| `branching.rs` | Branch create / list / delete, lazy fork |
+| `branching.rs` | Branch create/list/delete and lazy fork; native-control hardening includes main and named-source clone-only create recovery, invalid-name-before-clone, live path-prefix namespace rejection, legacy prefix-collision leaf-first delete, and delete/recreate first-write safety. The lower classifier truth cells (absent-ref/tree-present delete, same-identifier native refusal, recreated-identifier typed conflict with JSON details) live in `src/branch_control.rs` unit tests |
 | `merge_truth_table.rs` | Merge-pair truth table (MR-786): all 9×9 `(left_op, right_op)` cells from `{noop, addNode, removeNode, addEdge, removeEdge, setProperty, dropProperty, addLabel, removeLabel}`. Adding a new op to `OpVariant` forces a compile error in `build_case` until the new row + column are dispositioned. 36 executable cells run through real `branch_merge` with a structured oracle (`MergeOutcome` / `MergeConflictKind` + graph-state assert); 45 cells involving `dropProperty`/`addLabel`/`removeLabel` are recorded as `Unsupported` until the mutation grammar grows. |
 | `merge_fast_forward.rs` | Fast-forward branch-merge cost + correctness: an append-only adopted-source merge routes *new* rows through `stage_append` instead of one whole-delta `stage_merge_insert` (the full-outer join that exhausted the DataFusion memory pool on embedding-bearing tables). The regression gate is structural — it asserts WHICH staged-write primitive the merge invokes via the task-local write probes (`omnigraph::instrumentation`), not a brittle size threshold; also: merge yields source state, defers vector indexes to the reconciler, streams blob columns |
 | `writes.rs` | Direct-publish writes: cancellation, RFC-022 non-strict full-attempt reprepare from fresh branch authority, strict stale-write conflicts, multi-statement atomicity, MR-794 staged-write rewire (D₂ rejection, insert+update coalesce, multi-append coalesce, partial-failure recovery, load RI/cardinality recovery); the lance#7444 row-id-overlap regression (`filtered_read_after_merge_update_and_delete_keeps_row_ids_consistent` — merge-load → same-key merge-load → delete → keyed point lookup, green only under the vendored lance-table patch — plus its append-only control) |
 | `staged_writes.rs` | TableStore staged-write primitives (`stage_append`, `stage_merge_insert`, `commit_staged`, `scan_with_staged`, `count_rows_with_staged`) — primitive-level only; engine code uses the in-memory `MutationStaging` accumulator instead |
 | `forbidden_apis.rs` | Defense-in-depth source-walk guard: engine code (`exec/`, `db/omnigraph/`, `loader/`, `changes/`) must not reach around the sealed storage trait to Lance inline-commit APIs, nor open datasets directly (`Dataset::open` / `DatasetBuilder::from_uri`/`from_namespace`) — reads route through `Snapshot::open` and the held-handle cache; `// forbidden-api-allow: <reason>` sentinel exempts reviewed lines |
 | `failpoint_names_guard.rs` | Source-walk guard (same defense-in-depth shape as `forbidden_apis.rs`): every failpoint call site (`maybe_fail`, `ScopedFailPoint::new`/`with_callback`, `Rendezvous::park_first`) must reference a `failpoints::names` const, never a bare string literal — a typo'd literal compiles and silently never fires |
-| `lance_surface_guards.rs` | Pins the Lance API surfaces omnigraph depends on (named runtime + compile-only guards; see [lance.md](lance.md)) — the first smoke check on any Lance version bump; e.g. `compact_files_succeeds_on_blob_columns` pins blob-v2 compaction working (a red means a Lance regression) |
+| `lance_surface_guards.rs` | Pins the Lance API surfaces omnigraph depends on (named runtime + compile-only guards; see [lance.md](lance.md)) — the first smoke check on any Lance version bump; e.g. `compact_files_succeeds_on_blob_columns` pins blob-v2 compaction working, while Guard 9 proves an unlisted clone-only branch blocks raw create and `force_delete_branch` reclaims it, but also pins Lance's intentional no-tree-cleanup behavior when a slash-name path-child is live (a red means a Lance regression) |
 | `warm_read_cost.rs` | Cost-budget tests for the warm read path (query-latency work), measured at the object-store boundary with Lance `IOTracker` (the LanceDB IO-counted pattern): a warm same-branch read does 0 manifest opens, 1 version probe, validates the schema once (Fix 1 / finding A / Fix 2 at commit-history depth); stale same-branch reads perform exactly 2 probes and refresh manifest-only; recreated non-main branches with the same Lance version refresh by incarnation; recreated branch-owned table handles are distinguished by table e_tag or refresh-time cache clearing; recreated traversal topology is protected by per-edge-table e_tag in the graph-index cache key or refresh-time cache clearing; a warm *repeat* read does 0 table opens via the held-handle cache and a write re-opens only the changed table at its new version/e_tag (Fix 3/6A). Also the CSR topology-build cost guards: `fresh_branch_traversal_reuses_main_graph_index` (A1 — a lazy-fork branch reuses main's cached CSR index, 0 rebuilds via `graph_build_count`) and `single_edge_query_builds_only_referenced_edge` (A2 — a one-edge query builds only that edge via `graph_edges_built`); both force CSR via the scoped `with_traversal_mode` seam, so they need no `#[serial]`. See "Cost-budget tests" below. |
 | `write_cost.rs` | Cost-budget tests for the WRITE path (RFC-013), the latency twin of `warm_read_cost.rs` on the **shared `helpers::cost` harness** (`measure`/`IoCounts`/`assert_flat`/`local_graph`). Runs on **local FS**; gates the **internal-table** term (`__manifest` scans flat in commit-history depth, lineage rows included — `internal_table_scans_are_flat_in_history`, now **green every-PR** since RFC-013 step 2 brought the internal tables into `optimize`; the test compacts at each depth before measuring) plus green every-PR guards (single-insert `data_writes` bounded, a per-write read-op ceiling that fails the moment a round-trip is added, and a `measure_with_staged` fitness assert that a keyed insert routes through `stage_merge_insert` once with no `stage_append`/vector-index build). The **data-table opener** term is S3-only — see `write_cost_s3.rs` and the backend-split note in "Cost-budget tests" below |
 | `write_cost_s3.rs` | Bucket-gated (skips without `OMNIGRAPH_S3_TEST_BUCKET`) twin of `write_cost.rs` on the same `helpers::cost` harness: gates the **data-table opener** term (per-write latest-version resolution flat across commit depth on a real object store — per-version GETs are invisible on local FS). A cost gate, not a correctness test — run on demand, not in the every-merge `rustfs_integration` job (see the backend-split note in "Cost-budget tests" below) |
@ -51,8 +51,8 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav
 | `validators.rs` | Schema constraint enforcement (enum, range, unique, cardinality) across JSONL load, mutation insert/update. ALL THREE write surfaces — mutation, bulk load, AND merge — route through the unified `crate::validate` evaluator (Δ-scoped, index-backed, reusing these leaf checks). Cross-version-uniqueness closure: `cross_version_unique_rejected_on_mutation_insert` + `reinsert_existing_key_is_upsert_not_unique_violation` (mutation path); `cross_version_unique_rejected_on_append_load` + `merge_load_reupsert_existing_key_is_not_unique_violation` (load path). Per-table `Overwrite`: `overwrite_load_validates_ri_against_new_image` (an edges-only overwrite still resolves RI against retained committed nodes) + `append_load_rejects_orphan_edge`. The evaluator's own unit tests live in `src/validate.rs` (`#[cfg(test)]`); its merge-conflict equivalence is pinned by `merge_truth_table.rs` (OrphanEdge) + `branching.rs` (Unique/Cardinality merge tests). Intra-batch duplicate-`@key` rejection on every load mode is pinned by `consistency.rs::loader_rejects_intra_batch_duplicate_keys`; the mutation-coalesce counterpart (insert+update / chained updates of one id are NOT a self-collision) by `writes.rs`. Non-String `@unique` columns probe committed state with a TYPED literal (not a stringified key): `cross_version_unique_rejected_on_date_column` + `noncolliding_write_to_date_unique_column_succeeds` (a `Date @unique` collision is a proper `@unique` violation, and a distinct value does not raise a Date32-vs-Utf8 coercion error). Cardinality is keyed by edge id, last-wins (matching commit's `dedupe_merge_batches_by_id`): `merge_load_edge_src_move_rechecks_vacated_src_cardinality` (a Merge-load moving an edge recounts the vacated src for `@card` min) + `merge_load_duplicate_edge_id_counts_once_per_card` (a dup edge id under two srcs in one batch counts once, no spurious max violation). Direct deletes capture the ids they remove (from the delete op's own scan) into the change-set's `deleted_ids`, so a delete emptying a src is validated: `mutation_delete_edge_below_card_min_rejected` (a `delete Edge` dropping a src below `@card` min is rejected, not silently committed). |
 | `merge_cost.rs` | Cost-budget tests for branch MERGE on the shared `helpers::cost` harness: `merge_validation_is_delta_scoped` (a 1-row-delta merge opens ≤3 data tables — Δ-scoped, not the whole catalog; was ~6 pre-#5) and `merge_manifest_cost_grows_with_history` (the cross-branch `__manifest` open amplification still grows with commit depth — a separate, not-yet-addressed term — while validation `data_open_count` stays flat) |
 | `policy_engine_chassis.rs` | Engine-layer Cedar enforcement (MR-722): allow + deny through every `_as` writer via the SDK directly — no HTTP — proving embedded and CLI callers hit the same gate as the server, with action × scope shapes matching `authorize_request` |
-| `maintenance.rs` | `optimize` (compaction), `repair` (explicit uncovered-drift publish), and `cleanup` (version GC): empty/idempotent/no-op edges, policy validation, head preservation; `optimize` publishes its own compaction (`optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds`), skips pre-existing uncovered drift (`optimize_skips_preexisting_manifest_head_drift`), refuses to run while a `__recovery` sidecar is pending (`optimize_defers_when_recovery_sidecar_is_pending`), and compacts blob-v2 tables through the normal path (`optimize_compacts_blob_table_alongside_plain_table`); `repair` previews/heals verified maintenance drift, refuses raw semantic drift without `--force`, and forced repair publishes only by explicit operator choice; the index reconciler (iss-848): `index_build_tolerates_null_vector_rows` (logical load succeeds without inline index work; an untrainable Vector column then defers during reconciliation) and `optimize_materializes_index_declared_but_unbuilt` (optimize creates a declared-but-deferred index) |
-| `failpoints.rs` | Failure-injection coverage (gated on `failpoints` feature). RFC-022 includes deterministic post-stage/pre-effect races for mutation/load uniqueness and strict disjoint-head changes, plus the cross-handle post-effect `RecoveryRequired` → read-write-open rollback cell. Control/recovery race cells include `first_touch_post_create_open_error_keeps_recovery_ownership`, `branch_delete_orphans_sidecar_armed_after_initial_barrier`, `branch_merge_fences_target_delete_recreate_aba`, `branch_merge_fences_concurrent_sync_on_same_handle`, `branch_merge_rejects_fresh_target_manifest_change_before_effects`, `branch_merge_rechecks_late_sidecar_after_table_gates`, `cleanup_rechecks_sidecars_under_gc_gates`, and `full_recovery_rereads_sidecar_body_after_discovery`. The suite also includes the five per-writer effect → manifest-CAS recovery tests, write-entry in-process heal contract, storage-fault matrix, S3 recovery twin, and convergence-idempotent roll-forward regression. |
+| `maintenance.rs` | `optimize` (compaction), `repair` (explicit uncovered-drift publish), and `cleanup` (version GC): empty/idempotent/no-op edges, policy validation, head preservation; cleanup pins exact keep-count behavior (including keep larger than history), count/time retention of a live lazy branch, the oldest of multiple lazy pins, graph-wide fail-closed ordering on an unopenable pin, and refusal of uncovered main HEAD drift before any GC; `optimize` publishes its own compaction (`optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds`), skips pre-existing uncovered drift (`optimize_skips_preexisting_manifest_head_drift`), refuses to run while a `__recovery` sidecar is pending (`optimize_defers_when_recovery_sidecar_is_pending`), and compacts blob-v2 tables through the normal path (`optimize_compacts_blob_table_alongside_plain_table`); `repair` previews/heals verified maintenance drift, refuses raw semantic drift without `--force`, and forced repair publishes only by explicit operator choice; the index reconciler (iss-848): `index_build_tolerates_null_vector_rows` (logical load succeeds without inline index work; an untrainable Vector column then defers during reconciliation) and `optimize_materializes_index_declared_but_unbuilt` (optimize creates a declared-but-deferred index) |
+| `failpoints.rs` | Failure-injection coverage (gated on `failpoints` feature). RFC-022 includes deterministic post-stage/pre-effect races for mutation/load uniqueness and strict disjoint-head changes, plus the cross-handle post-effect `RecoveryRequired` → read-write-open rollback cell. Native controls are pinned by `native_branch_controls_reclassify_lost_acknowledgements` (matching create and absent-ref delete, with no version/lineage movement); `armed_first_touch_recovery_accepts_missing_target_ref` additionally forges and reclaims the clone-only/no-`BranchContents` table state. Legacy path overlap has both sides pinned: `armed_first_touch_recovery_defers_legacy_path_overlap_until_leaf_delete` permits open only for a proven no-effect intent, while `partial_first_touch_recovery_fails_closed_on_legacy_path_overlap` leaves one exact multi-table effect and verifies open fails closed until offline leaf cleanup lets rollback converge. Other control/recovery race cells include `first_touch_post_create_open_error_keeps_recovery_ownership`, `branch_delete_orphans_sidecar_armed_after_initial_barrier`, `branch_merge_fences_target_delete_recreate_aba`, `branch_merge_fences_concurrent_sync_on_same_handle`, `branch_merge_rejects_fresh_target_manifest_change_before_effects`, `branch_merge_rechecks_late_sidecar_after_table_gates`, `cleanup_rechecks_sidecars_under_gc_gates`, and `full_recovery_rereads_sidecar_body_after_discovery`. The suite also includes the five per-writer effect → manifest-CAS recovery tests, write-entry in-process heal contract, storage-fault matrix, S3 recovery twin, and convergence-idempotent roll-forward regression. |
 | `failpoint_names_guard.rs` | Source-walk guard: every `maybe_fail(...)` call site (engine + cluster) must reference the compile-checked `failpoints::names` consts, never a bare string literal — a typo'd literal compiles but silently never fires; same defense-in-depth shape as `forbidden_apis.rs` |
 | `recovery.rs` | Open-time recovery sweep — legacy sidecars plus schema-v3 Mutation/Load exact transaction identity, fixed logical/rollback outcome IDs, branch-token comparison, fresh under-gate sidecar reread/reparse, all-or-nothing roll-forward/rollback/refusal, audit row in `_graph_commit_recoveries.lance`, and `OpenMode::ReadOnly` skip path |
 | `composite_flow.rs` | Compositional/narrative end-to-end stories — multi-step flows that compose mechanics covered by other test files. Catches integration regressions where individual operations all pass their unit tests but their composition breaks (sequential merges, post-merge main writes, time-travel through merge DAG, reopen consistency over multi-merge histories, post-optimize and post-cleanup strict writes). |
--- a/docs/dev/writes.md
+++ b/docs/dev/writes.md
@ -30,13 +30,16 @@ authority.
  never created without ownership: the schema-v3 sidecar is durable first and
  names that `(table_path, target ref)`. Reclaim and `cleanup` treat any
  matching pending sidecar as a hard stop. Quiesced full recovery accepts both
-  crash shapes — sidecar durable with no ref yet, or an exact untouched ref at
-  the inherited version — and removes the latter before deleting the empty
-  intent. If several pending intents claim one ref, a no-effect intent discards
-  only itself while any competitor remains; the last no-effect survivor cleans
-  an untouched ref, or the effect-owning survivor recovers normally. `cleanup`
-  remains the backstop for genuinely unclaimed legacy/stale refs; see the
-  fork-reclaim note in [invariants.md](invariants.md).
+  logical crash shapes — sidecar durable with no ref yet, or an exact untouched
+  ref at the inherited version. Because Lance creates a branch dataset before
+  writing its authoritative `BranchContents`, the first shape may still contain
+  a clone-only tree; recovery force-reclaims that absent-ref tree idempotently.
+  It removes an exact untouched ref before deleting the empty intent. If several
+  pending intents claim one ref, a no-effect intent discards only itself while
+  any competitor remains; the last no-effect survivor cleans an untouched ref,
+  or the effect-owning survivor recovers normally. `cleanup` remains the
+  backstop for genuinely unclaimed legacy/stale refs; see the fork-reclaim note
+  in [invariants.md](invariants.md).

 ## Mutation/load coarse OCC (RFC-022 first adapter)

@ -87,6 +90,14 @@ source and target, re-lists sidecars, and compares fresh source/target manifest
 versions with the captured snapshots. A stale warm handle catalog or coordinator
 snapshot is never accepted as that revalidation.

+The source snapshot is a captured merge input, not authority that the target
+manifest CAS can arbitrate. The current process-local source gate is a stronger
+same-process fence around that capture, including delete/recreate ABA, but the
+semantic contract is still "merge the captured source commit." A later source
+advance does not invalidate an otherwise prepared target publish. Claiming
+"latest source at target publish" would instead require a cross-process source
+fence held through the target CAS.
+
 That fence prevents a same-process target delete/recreate from reusing the branch
 name underneath a merge plan. The race test deliberately recreates a target with
 the same name and numeric Lance version but a different `BranchIdentifier`, so
@ -109,6 +120,47 @@ records the orphan-discard recovery audit and deletes the sidecar. A
 is specific to removing the authority that made the intent reachable; create,
 merge, mutation, and load still reject relevant unresolved ownership.

+### Native graph-branch control recovery
+
+Graph branch create/delete do not use the graph-visible table-effect sidecar or
+emit graph lineage. Their sole logical authority is Lance `BranchContents` for
+the `__manifest` dataset, and Lance mutates that authority in two physical
+phases:
+
+- create shallow-clones `tree/{branch}` before writing `BranchContents`;
+- delete removes `BranchContents` before reclaiming that tree.
+
+Under the schema/branch/table control gates, create validates the name before
+the clone and rejects a live graph name that is a physical path ancestor or
+descendant of another live name. It then force-reclaims any absent-ref same-name
+tree and performs at most two native attempts. An ambiguous result is accepted
+only when fresh metadata has
+the captured parent branch/version/incarnation plus exactly one new identifier
+element and the target opens. Foreign or broken authoritative refs are never
+deleted. Delete captures the exact target identifier; after an ambiguous error,
+an absent ref is logical success, the same identifier preserves the original
+error, and a different identifier is a typed delete/recreate conflict. Derived
+tree cleanup is retried best-effort.
+
+There is deliberately no branch-control sidecar: within the supported
+single-writer-process topology, an absent ref makes a same-name tree unreachable
+garbage; the path-prefix-disjoint namespace is what makes Lance's recursive
+force cleanup exact. Same-name create is therefore the targeted reconciler.
+First-touch data-table
+forks remain sidecar-owned because they are physical effects of a graph-visible
+mutation/load. Lance does not expose conditional ref create/delete, so this
+classifier is not advertised as a cross-process branch-control fence.
+
+Legacy prefix-overlap recovery is the one first-touch case that does not prove an
+entire nested tree unreachable. If a Full sweep finds an ancestor first-touch
+target with a live path-child, it keeps the sidecar. Open may complete for
+leaf-first deletion only when the sidecar owns no physical table effect. A mixed
+attempt that owns an effect plus an untouched fork must roll back as one recovery
+outcome, so open fails closed while the child blocks fork cleanup. After an
+existing handle or an offline Lance-level branch tool removes the child, a later
+Full sweep reclaims the untouched fork, compensates the owned effect, and retires
+the intent.
+
 ## Read-your-writes within a multi-statement mutation

 A `.gq` query with multiple ops (e.g. `insert Person … insert Knows …`)
@ -328,12 +380,17 @@ recovery sweep in `crates/omnigraph/src/db/manifest/recovery.rs`:
  rollback-only; an unknown/foreign effect is refused rather than adopted.
  An Armed first-touch intent with no owned transaction is deferred by live
  roll-forward-only healing because another handle may still own it. Quiesced
-  full recovery tolerates an absent target ref (crash before fork), or removes
-  an exact unchanged unpublished ref after proving no other pending sidecar
-  claims it. With competing claims, the current no-effect sidecar discards
-  itself without touching the ref; the final survivor owns cleanup/recovery.
+  full recovery tolerates an absent target ref (either crash before clone or a
+  clone-only tree with no `BranchContents`) and force-reclaims that absent-ref
+  target idempotently. If an authoritative ref exists, recovery removes it only
+  when it is exactly unchanged and no other pending sidecar claims it. With
+  competing claims, the current no-effect sidecar discards itself without
+  touching the ref; the final survivor owns cleanup/recovery.
  During partial rollback, no-effect refs are removed before the rollback
-  outcome is published so a retry cannot strand them.
+  outcome is published so a retry cannot strand them. If a legacy live
+  path-child blocks that cleanup, rollback returns an error and read-write open
+  fails closed; only a sidecar proven to own no table effect may defer cleanup
+  while returning an open handle.
 - If any table is `InvariantViolation` (Lance HEAD < manifest pinned —
  should be impossible), **abort** with a loud error and leave the
  sidecar on disk for operator review.
--- a/docs/rfcs/rfc-022-unified-write-path.md
+++ b/docs/rfcs/rfc-022-unified-write-path.md
@ -446,7 +446,15 @@ able to enumerate every adapter and every entry point that invokes it.

 ### 6.2 Branch merge

- Compute row classification outside the gates.
+- Capture the exact source graph commit/snapshot and compute row classification
+  against that immutable input outside the effect gates. The source commit is an
+  effect precondition, not a member of the target publisher's atomic `ReadSet`:
+  a target CAS cannot arbitrate a foreign source-branch row.
+- Revalidate that the captured source incarnation and commit still identify the
+  prepared input before effects. A later source-head advance is intentionally
+  harmless: merge means "merge this captured source commit," not "whatever is
+  latest when the target publishes." A future latest-at-publish contract would
+  require a real source-branch fence held through target CAS.
 - Include the target graph head and every target table used by classification or
  validation in `ReadSet`.
 - Any target change before effects forces a complete reclassification; publishing a
@ -495,15 +503,82 @@ There is no target branch on which to publish before create, and no target remai
 which to publish after delete. They therefore cannot truthfully be instances of the
 graph-visible manifest-CAS protocol.

+The native control is not one physical mutation. At the pinned Lance revision:
+
+- create first commits a shallow-cloned branch dataset under `tree/{branch}` and
+  then writes `BranchContents`; the two phases are non-atomic and
+  `BranchContents` is the sole logical authority;
+- delete removes `BranchContents` first and then reclaims the branch directory.
+
+Accordingly, a clone without `BranchContents` is unreachable physical garbage,
+while an absent `BranchContents` after delete means the logical delete succeeded
+even if directory cleanup returned an error.
+
+Lance maps slash-separated branch names into nested physical directories and will
+not reclaim an ancestor directory while a live descendant name exists. Live graph
+branch names are therefore path-prefix-disjoint: `review/2026` is valid, but it
+cannot coexist with a live `review` branch. This is checked before native create,
+and legacy overlapping names must be deleted leaf-first. Without this namespace
+invariant, `force_delete_branch` may return success while deliberately leaving an
+ancestor clone-only dataset in place.
+
 Their control protocol is:

 1. run and await the recovery barrier;
 2. quiesce enrolled streams as required by RFC-026;
 3. acquire any active global claim, such as RFC-025's retention claim, and then
   the graph/branch-control gate in §4.3 order;
-4. freshly revalidate source ref, target existence, and branch incarnation;
-5. perform one native Lance ref mutation, which is the visibility point;
-6. release the gate and reclaim orphaned per-table forks asynchronously.
+4. freshly capture source ref/version/incarnation and target absence (or the
+   delete target's exact `BranchIdentifier`);
+5. validate a create name and the path-prefix-disjoint namespace before Lance's
+   shallow-clone phase;
+6. execute the bounded native classifier below; and
+7. reclaim owned per-table forks best-effort while the complete control-gate set
+   remains held, then release the gates. A reclaim failure is logged for the
+   cleanup reconciler; this implementation does not schedule asynchronous reclaim.
+
+| Operation | Fresh physical state | Classifier action |
+|---|---|---|
+| create | no ref, no clone | idempotent force-reclaim (no-op), then create |
+| create | clone only | force-reclaim the unreachable clone, then create |
+| create | matching ref + openable clone observed after this invocation's ambiguous native result | complete; accept a lost acknowledgement |
+| create | mismatching/broken ref | conflict; never adopt or delete it |
+| delete | captured identifier still present | not deleted; return the native error |
+| delete | ref absent, directory remains | logically deleted; retry reclaim best-effort |
+| delete | ref and directory absent | complete |
+| delete | different identifier present | delete/recreate ABA; conflict, never delete it |
+
+Create retries the native call at most once after reclaiming an absent-ref tree.
+It never adopts a matching ref that was already present before the invocation;
+that remains ordinary `AlreadyExists` because there is no operation marker.
+Matching completion requires the expected parent branch and version plus a target
+identifier whose mapping is exactly the captured parent identifier followed by one
+new `(parent_version, uuid)` element; the target dataset must also open. Lance does
+not let the caller pre-mint that UUID, so the proof is intentionally scoped to the
+supported single-writer-process boundary.
+
+No separate graph-branch sidecar is written. Under the held target gate,
+path-prefix-disjoint namespace, and supported topology, absent `BranchContents`
+means there is no logical branch and any same-name tree is safe derived garbage;
+the next same-name create is the targeted, idempotent reconciler. A recovery
+sidecar would introduce a second authority and would incorrectly pull native
+controls into graph lineage. First-touch **data-table** forks are different: their
+mutation/load sidecar is already durable before branch creation. Full recovery
+force-reclaims an absent-ref target as either a no-op (crash-before-clone) or that
+intent's clone-only zombie before retiring the intent.
+
+One compatibility case deliberately defers that cleanup. A legacy graph may
+already contain path-prefix-overlapping live names. If an unresolved first-touch
+intent targets an ancestor table tree while a live path-child still exists, Full
+recovery leaves the sidecar durable instead of recursively deleting the child's
+storage. Read-write open may complete for leaf-first remediation only when the
+intent owns no physical table effect. If a multi-table attempt owns any effect
+while another untouched fork is blocked by the child, rollback is
+complete-or-error: open fails closed, the sidecar remains authoritative, and an
+existing handle or offline Lance-level branch tool must remove the descendant
+before the next Full recovery reclaims the untouched fork, compensates the owned
+effect, and retires the sidecar. This distinction prevents legacy writers outside
+the v3 barrier from preparing against an unresolved partial rollback.

 Delete has one recovery disposition that create does not: after the complete
 schema/target-branch/all-table gate set has waited out any live in-process owner,
@ -515,13 +590,14 @@ create/merge may not adopt this exception.
 The native ref operation itself should enforce the freshly checked precondition or
 surface concurrent ref mutation as a conflict — but at the pinned Lance revision it
 does not: branch-ref creation is an existence check followed by an unconditional
-put, not a conditional primitive (the same fact for which RFC-025 §2.3 rejects a
-branch ref as a claim mechanism). Until Lance ships a conditional/CAS ref mutation,
-graph-branch create/delete therefore inherit the documented single-writer-process
-support boundary — the same disposition RFC-023 §10 applies to recovery ownership —
-and multi-process branch operations are not advertised. The upstream ask for a
-conditional ref primitive is filed alongside this RFC; a process-local branch gate
-remains a local optimization, not the missing cross-process guarantee.
+put, and delete is not compare-and-delete by `BranchIdentifier` (the same substrate
+gap for which RFC-025 §2.3 rejects a branch ref as a claim mechanism). Until Lance
+ships conditional/CAS ref mutation, graph-branch create/delete therefore inherit the
+documented single-writer-process support boundary — the same disposition RFC-023
+§10 applies to recovery ownership — and multi-process branch operations are not
+advertised. The upstream ask for a conditional ref primitive is filed alongside
+this RFC; a process-local branch gate remains a local optimization, not the missing
+cross-process guarantee.

 These operations do not emit a synthetic graph commit. If a future product contract
 requires a native ref mutation and manifest/audit rows to become atomic together, it
@ -628,6 +704,16 @@ Implementation proceeds in this order:
   > its full exact-effect adapter remains future work. Schema apply,
   > optimize/index, and MemWAL fold remain on their writer-specific paths until
   > their adapter slices land.
+
+   > **Branch-control/cleanup slice (2026-07-11):** native graph-branch create
+   > and delete now use the §7 authority classifier, including pre-clone name
+   > validation, prefix-disjoint live names, bounded clone-only recovery,
+   > lost-acknowledgement classification, exact delete-incarnation fencing, and
+   > no synthetic graph lineage. First-touch Full recovery also reclaims an
+   > absent-ref clone-only table fork. Cleanup now protects every exact lazy
+   > branch pin, computes `keep` from Lance's actual version list, refuses
+   > uncovered main HEAD drift, and completes its live-root preflight before the
+   > first table GC.
 3. Convert mutation/load, branch merge, schema apply/migration, data-table optimize,
   and graph-visible index work one adapter at a time.
 4. Add static or runtime enumeration proving no graph-visible entry point bypasses the
@ -661,6 +747,10 @@ writers.
 - An edge insert racing deletion of an endpoint must revalidate or conflict.
 - Cardinality probes of an untouched table participate in arbitration.
 - A target advance after merge classification forces complete reclassification.
+- A source branch advancing after capture does not silently substitute its new
+  head: the merge either uses the captured source commit or fails its pre-effect
+  source-incarnation/commit check. A latest-at-target-publish variant requires a
+  real source fence and is not inferred from the target `ReadSet`.
 - Run the same cells with separate `Omnigraph` handles sharing one root-scoped
  process-local gate manager, then with separate processes that do not share it.

@ -685,6 +775,19 @@ writers.
  retryable physical contention.
 - MemWAL fold, when RFC-026 lands: merged-generation conflict and every fold crash
  boundary.
+- Native graph branch control: invalid name before clone; live path-prefix
+  collision before clone; clone-only create recovery on main and a named source;
+  pre-existing matching ref remains `AlreadyExists`; current-call lost create
+  acknowledgement is accepted only with exact parent/incarnation metadata and an
+  openable target; delete with absent ref plus remaining tree is success and
+  reclaims best-effort; same identifier preserves the native error; a recreated
+  identifier is a typed conflict and survives; neither direction advances a
+  manifest version or emits graph lineage.
+- First-touch recovery with legacy path overlap: a proven no-effect intent may
+  defer clone-only ancestor cleanup and return an open handle for leaf-first
+  remediation; a mixed multi-table intent with one owned effect and one blocked
+  untouched fork must fail read-write open, retain the sidecar without restoring
+  or publishing, and converge after offline Lance-level leaf cleanup.

 ### 11.5 Cost gates

--- a/docs/rfcs/rfc-025-checkpoint-retention.md
+++ b/docs/rfcs/rfc-025-checkpoint-retention.md
@ -274,6 +274,16 @@ The cleanup root set is:
 - every version inside the operator-selected time/count retention window;
 - versions Lance itself protects through non-OmniGraph tags or branch refs.

+Checkpoint tags do not physically protect a lazy graph branch whose table row
+still points at an exact version on that data table's `main`: Lance cannot see
+the foreign `__manifest` reference. For each physical main dataset, cleanup
+therefore caps `before_version` at the oldest exact inherited-main version among
+all live graph branches. Native per-table branch refs remain Lance-protected and
+do not need duplicate tags. Failure to resolve or open any live root aborts the
+graph-wide preflight before the first table GC. The current shipped baseline also
+requires each manifest-visible main version to equal Lance HEAD; uncovered drift
+must be repaired before retention work.
+
 Read-only preview may run without a claim, but it is explicitly provisional.
 Confirmed execution uses this order:

@ -283,7 +293,8 @@ Confirmed execution uses this order:
 2. acquire the retention claim, then revalidate the fleet outage, every stream's
   `SEALED` cut, and absence of recovery sidecars;
 3. run pin reconciliation and recompute the exact root set and per-dataset
-   cutoffs under the claim;
+   cutoffs under the claim, including the oldest live lazy-branch inherited-main
+   floor above;
 4. prepare and revalidate a complete RFC-022 `ReadSet` containing format/schema
   identity, branch incarnations, current manifest table entries/heads, prior GC
   boundaries, and the root digest;
--- a/docs/user/branching/index.md
+++ b/docs/user/branching/index.md
@ -6,12 +6,34 @@ Lance supports branching at the dataset level: a branch is a named lineage of ve

 ## L2 — Graph-level branches

-OmniGraph builds *graph branches* on top by branching every sub-table coherently:
+OmniGraph builds *graph branches* on top with one authoritative `__manifest`
+ref whose table entries form a coherent graph snapshot; data-table forks are
+created lazily on first write:

- **Create** (`branch create` / `branch create --from <target>`) — the name `main` is disallowed; fails if the branch exists. Atomic: the new branch becomes visible all-or-nothing, so a name never half-exists.
+- **Create** (`branch create` / `branch create --from <target>`) — the name `main` is disallowed; fails if the branch exists. Logically atomic: the branch becomes visible only when its authoritative ref exists, so a name never half-exists in graph reads. If storage interruption leaves only Lance's unreachable shallow-clone directory, the next same-name create reclaims it and retries automatically.
 - **List** (`branch list`) — returns public branches, **filtering the internal** `__schema_apply_lock__` branch.
- **Delete** (`branch delete`) — refuses if there are descendants on the branch, or if it is the current branch. Once deleted, the branch is gone from every snapshot. The owned per-table forks are reclaimed best-effort; if that reclaim hits a transient object-store error, the leftover storage is reclaimed later by the [`cleanup`](../operations/maintenance.md) command. One consequence: if a delete's reclaim fails, reusing that branch name before the next `cleanup` surfaces a clear error pointing at `cleanup`.
+- **Delete** (`branch delete`) — refuses if there are descendants on the branch, or if it is the current branch. Once its authoritative ref is removed, the branch is gone from every snapshot even if reclaiming a now-unreachable storage directory needs a later retry. Owned per-table forks are reclaimed best-effort; same-name create/first write safely reconciles relevant leftovers, and [`cleanup`](../operations/maintenance.md) remains the general backstop.
 - **Lazy forking**: a branch only forks a sub-table when that sub-table is first mutated on it. Pure-read branches share storage with their source. If two writers race to first-write the same branch, the loser gets a retryable "refresh and retry".
+- **Names are path-prefix-disjoint while live.** Slash-separated names are
+  supported (`review/2026-07`), but `review` and `review/2026-07` cannot both be
+  live because Lance stores them in overlapping physical directories. Choose
+  sibling names, or delete the existing ancestor/descendant first.
+
+Graph branch create/delete are coordinated across handles in one writer
+process. Until Lance exposes conditional native ref mutation, separate writer
+processes must not concurrently control branches on the same graph.
+
+For a legacy graph that already contains path-prefix-overlapping live names,
+recovery also preserves the leaf-first escape hatch. If read-write open finds an
+unresolved first-touch sidecar for an ancestor table fork while a live path-child
+remains, it never deletes the child's storage. When the interrupted write owns no
+table effect, it leaves the sidecar in place and completes the open so you can
+delete the descendant branch leaf-first. When the same sidecar owns a partial
+table effect, open fails closed because cleanup and rollback must finish together;
+remove the descendant through an already-open handle or an offline Lance-level
+branch tool, then reopen. The next read-write open reclaims the ancestor residue,
+rolls back the partial effect, and retires the sidecar. `omnigraph repair` is not
+that offline tool: it correctly refuses to run while a recovery sidecar is pending.

 ## L2 — Commit graph

--- a/docs/user/operations/maintenance.md
+++ b/docs/user/operations/maintenance.md
@ -30,28 +30,53 @@
 - Garbage-collects old versions per table.
 - Removes versions (and their unique fragments) older than the retention policy.
 - Policy options `keep_versions` and `older_than` — at least one is required.
+  `keep_versions=N` derives its requested cutoff from the newest `N` available
+  versions per table (the current HEAD is always retained, including for
+  `N=0`); live-branch safety floors may retain additional intervening versions.
+  When both options are set, a version is removed only if it is older than both
+  cutoffs.
 - Returns per-table stats: `table_key, bytes_removed, old_versions_removed, error`.
- **Fault-isolated per table.** A single table's transient failure (version GC or
-  orphan reclaim) is recorded on that table's stats row (with an `error`) and logged,
-  and never aborts the healthy tables — cleanup is the convergence
-  backstop, so it does as much as it can and converges on re-run. The CLI reports
-  any failed tables; rerun `cleanup` to retry them.
+- **Fault-isolated per table after the graph-wide safety preflight.** A single
+  table's transient version-GC failure is recorded on that table's stats row
+  (with an `error`), logged, and reported by the CLI without aborting healthy
+  tables. Orphan-reclaim failures are also logged and retried on a later cleanup,
+  but the current stats/CLI surface does not attach them to
+  `TableCleanupStats.error`. The recovery and live-branch checks below are
+  preflight invariants instead: either must succeed before any version GC runs.
+  Rerun `cleanup` to converge either kind of per-table failure.
 - CLI guards with `--confirm`; without it, prints a preview line.
 - **Non-local consent.** Against a non-local target (an `s3://` store/cluster), `cleanup` additionally requires `--yes` on top of `--confirm`: a TTY is prompted, and a non-interactive run (no TTY, or `--json`) refuses rather than destroying. A local (`file://`) target needs only `--confirm`. The same `--yes` gate applies to overwrite `load` and `branch delete`; every maintenance run echoes its resolved target to stderr (suppress with `--quiet`).
- **Recovery floor:** `--keep < 3` may garbage-collect versions that crash recovery needs as a rollback target. Default `--keep 10` is safe.
+- **Recovery floor:** `--keep < 3` may garbage-collect versions that a later
+  rollback would otherwise use. `--keep 10` is the recommended conservative
+  count; there is no implicit default, so pass a policy explicitly.
 - **Requires clean recovery state.** If any durable recovery intent is pending,
  cleanup refuses before orphan reconciliation or version GC. Reopen the graph
  read-write (or restart the server) to resolve recovery, then rerun cleanup;
  deleting transaction/version history while an intent is pending would make
  exact effect ownership unverifiable.
+- **Preserves live lazy branches.** A graph branch initially inherits each data
+  table directly from an exact main-table version; until that table is first
+  written on the branch, there is no native Lance data-table branch for Lance's
+  cleanup to discover. Under the same schema, branch, and table gates used for
+  version GC, cleanup therefore reads every live graph-branch snapshot and caps
+  each main dataset's cutoff at its oldest inherited version. Native per-table
+  forks remain protected by Lance itself. If any live branch snapshot cannot be
+  classified, cleanup refuses before garbage-collecting any table rather than
+  guessing that its referenced versions are disposable.
+- **Refuses uncovered main-table drift.** Every manifest-visible main version
+  must open and equal Lance HEAD during the graph-wide preflight. If an external
+  or interrupted operation advanced HEAD without a recovery sidecar, run
+  `omnigraph repair` before cleanup; version GC never guesses around that drift.
 - **Orphaned-branch reconciliation:** before the version GC, cleanup reclaims any per-table Lance branch absent from the manifest branch list. These orphans arise when a `branch_delete` flips the manifest authority but a downstream best-effort reclaim does not complete (see [branches-commits.md](../branching/index.md)). The reconciler is idempotent (it no-ops once nothing is orphaned), runs regardless of the `keep_versions` / `older_than` values (those gate version GC only), and never reclaims `main` or system-branch forks. Reclaimed forks are logged. Graph lineage has no separate branch dataset: it lives in `__manifest`.

 ## Tombstones

 Logical sub-table delete markers in `__manifest` that exclude a sub-table version from snapshot reconstruction.

-## Internal schema migrations
+## Internal schema versions

-Version evolutions of the on-disk `__manifest` shape are reconciled automatically on the first write under a new binary. An on-disk stamp records the shape; the binary migrates it forward before reading state, and reads are side-effect-free. No operator action is required for in-place upgrades. See [storage.md → Internal schema versioning](../concepts/storage.md) for the full mechanism.
-
-A binary opening a manifest stamped at a version *higher* than it knows about refuses to publish with a clear "upgrade omnigraph first" error — old binaries cannot clobber a newer schema.
+The on-disk format is strict-single-version. A binary refuses a graph whose
+internal schema stamp differs from the version it supports; storage-format
+changes use export/import rebuild rather than automatic in-place migration.
+See the [upgrade guide](upgrade.md) and
+[versioning policy](../../dev/versioning.md).