b6c19bfa5d
Patch release over v0.7.1: write-path latency reductions (#288 direct table opener, #298 schema-once + open-each-table-once) and three correctness fixes on the maintenance and recovery paths (#297 optimize survives a cross-process write race, #291 optimize compacts the internal metadata tables + non-destructive auto_cleanup strip, #296 recovery converges under a concurrent manifest advance). No breaking changes, no on-disk format change, no migration. Version coherence: all 7 crate manifests + path-dep constraints, Cargo.lock, openapi.json, and the AGENTS.md surveyed version bumped 0.7.1 -> 0.7.2. Build green under --locked; OpenAPI drift check green.
3.4 KiB
Omnigraph v0.7.2
A patch release over v0.7.1: write-path latency reductions plus three correctness fixes on the maintenance and recovery paths. No breaking changes, no on-disk format change, and no migration — drop-in over v0.7.1.
Performance
-
Write opens go direct, schema validates once (#288, #298). Write opens used to route through the per-table Lance namespace catalog, which re-opened the dataset just to read its location and re-resolved the latest version on every table open — an O(commit-depth) double resolution that dominated write latency on object stores (~70%). Writes now open each touched data table directly by its manifest-recorded location (Lance's O(1) version-hint path), validate the schema contract once per write instead of ~4×, and open each touched table once instead of 4×.
-
optimizecompacts the internal metadata tables (#291).optimizepreviously iterated only node/edge tables, so the internal__manifest,_graph_commits, and_graph_commit_actorstables accumulated one fragment per commit and were never compacted — making every write's metadata scan grow with commit history.optimizenow compacts all three, so a periodically optimized long-lived graph keeps its per-write metadata scan flat in history.
Fixes
-
optimizesurvives a cross-process write race (#297). A CLIoptimizeracing a served write on the same table could fail: the in-process write queue doesn't serialize across processes, so a concurrent insert/delete advancing the manifest between optimize's compaction and its publish broke the strict equality CAS. Optimize now reopens-and-replans on a genuine Lance conflict and fast-forwards its publish monotonically, so a maintenance compaction never fails a live write. Bounded retry; sustained contention surfaces a loud conflict rather than dropping work. -
optimizeis non-destructive on upgraded graphs (#291). A graph created by a pre-0.7.0 binary carries an on-by-default Lance auto-cleanup config; under it, optimize's compaction commit could fire Lance's version-GC hook and prune__manifest-pinned versions (breaking snapshots and time travel). Optimize now strips any stalelance.auto_cleanup.*config off every table — data and internal — before its HEAD-advancing commits, so compaction can never GC pinned versions. -
Recovery converges instead of failing
openunder a concurrent manifest advance (#296). The open-time recovery sweep published its roll-forward at the sidecar's pinned expected version; if another writer advanced the manifest during the classify→publish window, the CAS failed and aborted the wholeOmnigraph::open. The sweep now treats roll-forward as "the manifest reflects the sidecar's committed state," not "this sweep won the CAS": on a CAS loss it re-reads the live manifest and, when the sidecar's intent is already satisfied, records the recovery and deletes the sidecar idempotently — so a concurrent advance no longer fails the open. (The destructive roll-back twin still defers to a cross-process lease, as documented.)
Upgrade notes
Drop-in over v0.7.1 — no configuration, schema, or data changes. Upgrade the server and CLI together as usual. Graphs created on v0.7.1 read and write identically on v0.7.2; the optimize non-destructive fix additionally protects graphs created by pre-0.7.0 binaries from version GC during compaction.