On full re-read of MR-737, the section numbering used in several of the
original MR-925 cross-references no longer lines up:
- §5.10 in current MR-737 is 'First-class scores and rank fusion', NOT
'custom index types' / 'connector SPI'. The custom-index-type / plugin
surface is §5.4 ('Persisted CSR adjacency as Lance index plugin') with
the capability shape in §5.6.
- §5.11 is 'Substrate choice — DataFusion vs. custom executor (A)'
(resolved 2026-05-11), NOT 'per-table txn branches'. Per-table Lance
native txn branches live in §5.12 ('Mutation IR, write planner, and
external sources') per Ragnor 2026-04-29.
- §5.5 is 'Stable row IDs as graph IDs', NOT 'reconciler pattern'. The
reconciler is §5.16.
- §5.8 is 'Tiering via Lance base paths', NOT SIP-related. SIP is §5.3.
- The MR-925 §1.6 cross-reference to 'Open Q5' was to a pre-2026-05-11
numbering; Q5 in current §10 is 'extension rate under filters'.
Each writeup now has a §-numbering note at the top mapping its findings
to the current MR-737 numbering. The findings themselves are unchanged —
this is a numbering-only edit.
Co-Authored-By: Ragnor Comerford <ragnor.comerford@gmail.com>
- validation-prototypes/custom-operator/: NeighborExpand toy operator
with paired ExtensionPlanner + custom QueryPlanner via
SessionStateBuilder::with_query_planner
- writeup at .context/experiments/custom-operator.md: 5 probes
(round-trip, EXPLAIN, predicate guard, composition with Filter +
Aggregate, BaselineMetrics) \u2014 all pass; ~250 LoC integration
footprint; no unsafe; no internal API access
- finding: \u00a75.3 is achievable on DF 52.5 as written; deltas are
doc-shaped (predicate push-down opt-in, statistics requirement,
Partitioning override)
- exclude validation-prototypes/ and merge-insert-cas-repro from the main
workspace so the nested cargo workspace can use its own pin set
- add validation-prototypes/{factorized-batches,custom-lance-index}/
scratch crates (never merged to main; long-lived branch only)
- exp 1.1 — factorized batches through DataFusion ops: writeup at
.context/experiments/factorized-batches.md (5 cells × 8 ops; all
scalar-keyed ops accept List<UInt64> input, UNNEST via CROSS JOIN
fails in DF 52.5)
- exp 1.2 — custom Lance index plugin from outside lance: writeup at
.context/experiments/custom-lance-index.md (5 probes; transaction
surface is open, SCALAR_INDEX_PLUGIN_REGISTRY is closed → hard
blocker for MR-737 §5.4; recommends upstream path or external-index
path)
