omnigraph/crates
Ragnor Comerford d138902e7e fix(engine): retry publish on RetryableCommitConflict (compaction vs publish)
Step 2 compacts `__manifest` with no app-level lock (Lance OCC arbitrates,
validated against LanceDB + the lance-7.0.0 conflict resolver). compact_files'
`Operation::Rewrite` auto-retries 20x (CommitConfig default num_retries=20), so a
live publish usually wins the race and the compaction rebases. But the publish
runs its merge-insert with conflict_retries(0) = one rebase attempt; if the
compaction commits first AND the merge touched a fragment the Rewrite rewrote,
Lance preempts the publish with `Error::RetryableCommitConflict` — a DIFFERENT
variant from the row-level `TooMuchWriteContention` the publisher already retries.
Left unhandled, that surfaces a transient error to the caller, i.e. a maintenance
compaction (physical op) failing a live write (logical op) — invariant 7.

Map `LanceError::RetryableCommitConflict` to a new
`ManifestConflictDetails::RetryableCommitConflict` and treat it as retryable in the
publisher's outer loop (reload fresh state + re-merge), alongside
RowLevelCasContention. `ExpectedVersionMismatch` still propagates (a genuine
expectation break must not be blindly retried). This also hardens multi-process
concurrent writers generally, not just compaction.

Normal publishes are insert-only (new object_ids -> new fragments, disjoint from
rewritten old ones), so the conflict is rare; the guard covers the
same-fragment-update edge and multi-process writers. Unit tests in publisher.rs
pin the mapping + the retry-predicate contract.
2026-06-20 20:49:38 +02:00
..
omnigraph fix(engine): retry publish on RetryableCommitConflict (compaction vs publish) 2026-06-20 20:49:38 +02:00
omnigraph-api-types release: v0.7.1 (#290) 2026-06-19 23:12:44 +03:00
omnigraph-cli release: v0.7.1 (#290) 2026-06-19 23:12:44 +03:00
omnigraph-cluster release: v0.7.1 (#290) 2026-06-19 23:12:44 +03:00
omnigraph-compiler release: v0.7.1 (#290) 2026-06-19 23:12:44 +03:00
omnigraph-policy release: v0.7.1 (#290) 2026-06-19 23:12:44 +03:00
omnigraph-server release: v0.7.1 (#290) 2026-06-19 23:12:44 +03:00