MR-927 Phase 1 — stable-row-id repro across BTree/Bitmap/LabelList + compaction

Builds and runs the small repro specified in
.context/experiments/stable-row-id-compaction.md §5 ("Small repro plan")
and extends the writeup with the empirical Phase 1 evidence (F7–F11).

Matrix {BTree, Bitmap, LabelList} × {stable=true, stable=false}, 6 fragments
forced via small max_rows_per_file and target_rows_per_fragment, with
with_row_id() probes pre- and post-compaction. All six cases return correct
counts; with enable_stable_row_ids: true the row IDs round-trip unchanged
across compaction; with the flag off the row addresses move (fragment_id <<
32 | local_row), which is the documented contract.

Plus a side experiment confirming that Operation::Overwrite (both staged
via InsertBuilder::execute_uncommitted + CommitBuilder::execute and direct
Dataset::write Overwrite) inherits manifest.uses_stable_row_ids from the
existing dataset, even when the WriteParams flag is absent. This resolves
the suspicion about table_store.rs:956 (stage_overwrite path not setting
the flag): the path is correct, not a latent bug.

Conclusion: MR-737 §5.5 substrate caveat ("Stable Row ID for Index is
documented as experimental in lance-4.0.x") is empirically resolved.
Feature works; docs are conservative. RFC shape for MR-927 is a docs-PR.

Refs MR-925, MR-927.

Co-Authored-By: Ragnor Comerford <ragnor.comerford@gmail.com>
This commit is contained in:
Devin AI 2026-05-12 22:56:51 +00:00
parent c0d7ae6ba4
commit 3b661f35d3
5 changed files with 754 additions and 1 deletions

View file

@ -5,10 +5,10 @@ members = [
"custom-lance-index",
"custom-operator",
"sip-format-bench",
"stable-rowid-index", # 1.7 / MR-927 Phase 1
# Additional crates added as each experiment is set up:
# "bitmap-pushdown", # 1.5
# "txn-branches-cost", # 1.6
# "stable-rowid-index", # 1.7
]
# Pre-Phase-0 validation prototypes for MR-925 / MR-737.