build(deps): bump Lance 6.0.1 → 7.0.0 (correct-by-design substrate alignment) (#229)

* build(deps): bump Lance 6.0.1 → 7.0.0 (object_store 0.13.2, roaring 0.11.4)

Arrow stays 58 and DataFusion stays 53 (no change). The only transitive bump
is object_store 0.12.5 → 0.13.2. 141 upstream commits reviewed; no fixes lost
(the 6.0.x release-branch backports are all forward-ported into 7.0.0).

- object_store 0.13 moved get/put/head/rename/delete behind a new ObjectStoreExt
  trait (list/list_with_delimiter/put_opts stay on the core trait). Add
  `use object_store::ObjectStoreExt` in storage.rs and db/manifest/namespace.rs;
  no call-site changes. Mirrors Lance's own migration in PR #6672.
- roaring pinned to 0.11.4 (cargo update -p roaring --precise 0.11.4). Lance
  7.0.0's UpdatedFragmentOffsets newtype (lance#6650) derives Eq over
  HashMap<u64, RoaringBitmap>, which needs RoaringBitmap: Eq, added in roaring
  0.11.4; the loose `roaring = "0.11"` constraint otherwise resolves 0.11.3 and
  lance itself fails to compile.
- lance#6774: merge-insert INSERT rows now stamp _row_created_at_version with the
  commit version (was a fallback of 1). Flip the lance_version_columns assertion
  to `== v2` and correct the changes/mod.rs rationale comment. Production
  change-detection keys on _row_last_updated_at_version + ID membership, so its
  logic is unaffected.

Refs lance#6650, lance#6774, lance#6672.

* fix(storage): pin WriteParams::auto_cleanup = None (lance#6755 default flip)

lance#6755 flipped the WriteParams::auto_cleanup default from on (a full cleanup
pass every 20th commit) to None. On 6.0.1 the on-by-default hook could silently
GC versions that __manifest pins for snapshots/time-travel. OmniGraph owns
cleanup explicitly (optimize.rs::cleanup_all_tables) and never set auto_cleanup,
so it was relying on a default that is both wrong for our snapshot model and now
changed upstream.

Pin auto_cleanup: None explicitly at all 11 production WriteParams sites
(table_store ×6, commit_graph ×2, recovery_audit ×1, manifest/graph ×2 — the
__manifest + sub-table Create paths). Removes the dependency on a default-flag
value and locks in the snapshot-safe behavior regardless of future upstream
re-flips.

Refs lance#6755.

* test(lance): pin BTREE range-boundary correctness (lance#6796)

lance#6796 (issue #6792) fixed a BTREE scalar-index range-query bound
inclusiveness bug: `x <= hi AND x > lo` returned the wrong boundary row.
Add lance_surface_guards.rs::btree_range_query_boundary_is_correct, which
reproduces the exact #6792 shape (5 rows + an explicit BTREE drives the index
path even on tiny data) and pins the corrected inclusive-<= / exclusive->
semantics. It turns red if a future Lance regression reintroduces the bug.

OmniGraph today builds BTREE only on string @key columns and queries them by
equality/IN, so its current patterns do not hit this; the guard protects any
future BTREE-range path (BTREE-on-properties, range-on-key).

Refs lance#6796.

* docs(dev): align Lance docs + invariants to 7.0.0

- docs/dev/lance.md: new 2026-06-14 alignment stanza for the 6.0.1 → 7.0.0 bump
  (object_store ObjectStoreExt move, roaring 0.11.4, #6774/#6796/#6755 behavior,
  #6658 shipped → MR-A unblocked but separate, #6666 + blob compaction still
  open); prior 6.0.1 stanza demoted to historical.
- AGENTS.md: storage substrate 6.x → 7.x (line + architecture diagram).
- docs/dev/invariants.md: deletes/vector known gap updated — the staged
  two-phase delete API (lance#6658) now exists and MR-A is unblocked, but
  delete_where stays inline and D2 stays in place until the migration lands;
  create_vector_index still gated on lance#6666.

* fix(storage): skip Lance auto-cleanup on commit paths for legacy datasets

Addresses PR #229 review (Codex P1). `WriteParams::auto_cleanup` is create-time
config with no effect on existing datasets (Lance write.rs docs), so the previous
`auto_cleanup: None` change alone did NOT protect graphs created before the v7
bump: 6.0.1 defaulted auto_cleanup ON, leaving `lance.auto_cleanup.*` config on
those datasets, and Lance's per-commit hook (io/commit.rs: `if
!commit_config.skip_auto_cleanup`) fires off that stored config — so omnigraph's
own writes would GC versions the __manifest pins for snapshots/time-travel.

Skip the hook on every commit path, covering new and legacy datasets alike:
- commit_staged: CommitBuilder::with_skip_auto_cleanup(true) — the staged data path.
- __manifest publisher: MergeInsertBuilder::skip_auto_cleanup(true).
- all 11 WriteParams: skip_auto_cleanup: true (direct Dataset::write/append paths;
  auto_cleanup: None retained so new datasets store no cleanup config at all).

Tests:
- lance_surface_guards::skip_auto_cleanup_suppresses_version_gc — substrate:
  negative control (config GCs v1 without skip) + with-skip survival.
- staged_writes::commit_staged_skips_auto_cleanup_so_pinned_versions_survive —
  omnigraph usage: commit_staged on a legacy-config dataset preserves the pinned
  create version.

Refs lance#6755.

* test(lance): assert created_at-preserved + updated_at-bumped on merge_insert UPDATE

Addresses PR #229 review follow-up. `lance_merge_insert_update_preserves_created_at_version`
documented (in a comment) that a merge_insert UPDATE preserves created_at and
bumps updated_at, but only asserted the value change — leaving the change-feed
invariant unguarded. Add the two missing assertions:

- bob created_at == v1 (preserved across UPDATE; what the test name promises;
  lance#6774 only changed INSERT-row stamping).
- bob updated_at == v2 (bumped to the commit version) — the invariant
  OmniGraph's insert/update classification relies on (changes/mod.rs keys on
  _row_last_updated_at_version). A regression here would silently drop updates
  from the diff/change feed.

crates/omnigraph/src/changes/mod.rs

@@ -248,12 +248,12 @@ async fn diff_table_same_lineage(
     // Inserts + Updates: use _row_last_updated_at_version to find all rows
     // touched since Vf, then classify by checking whether the ID existed at Vf.
     //
-    // Why not _row_created_at_version for inserts: Lance's merge_insert stamps
-    // new rows with _row_created_at_version = dataset_creation_version (v1),
-    // not the merge_insert commit version. This makes _row_created_at_version
-    // unreliable for detecting inserts from merge_insert writes. Using
-    // _row_last_updated_at_version catches all touched rows regardless of
-    // write mode, and ID-set membership distinguishes inserts from updates.
+    // We key on _row_last_updated_at_version because one scan over it catches
+    // every row touched in the window — inserts and updates alike — regardless
+    // of write mode, and ID-set membership at Vf then distinguishes inserts from
+    // updates. (lance#6774 made merge_insert stamp new rows' _row_created_at_version
+    // with the commit version, so created_at became reliable too; last_updated
+    // stays the right key since it also covers updates.)
     if wants_inserts || wants_updates {
         let filter_sql = format!(
             "_row_last_updated_at_version > {} AND _row_last_updated_at_version <= {}",

crates/omnigraph/src/db/commit_graph.rs

@@ -57,6 +57,8 @@ impl CommitGraph {
             mode: WriteMode::Create,
             enable_stable_row_ids: true,
             data_storage_version: Some(LanceFileVersion::V2_2),
+            auto_cleanup: None,
+            skip_auto_cleanup: true,
             ..Default::default()
         };
         let dataset = Dataset::write(reader, &uri as &str, Some(params))
@@ -430,6 +432,8 @@ async fn create_commit_actor_dataset(root_uri: &str) -> Result<Dataset> {
         mode: WriteMode::Create,
         enable_stable_row_ids: true,
         data_storage_version: Some(LanceFileVersion::V2_2),
+        auto_cleanup: None,
+        skip_auto_cleanup: true,
         ..Default::default()
     };
     match Dataset::write(reader, &uri as &str, Some(params)).await {

crates/omnigraph/src/db/manifest/graph.rs

@@ -31,6 +31,8 @@ pub(super) async fn init_manifest_graph(
         mode: WriteMode::Create,
         enable_stable_row_ids: true,
         data_storage_version: Some(LanceFileVersion::V2_2),
+        auto_cleanup: None,
+        skip_auto_cleanup: true,
         ..Default::default()
     };
     let manifest_path = manifest_uri(root);
@@ -127,6 +129,8 @@ async fn create_empty_dataset(uri: &str, schema: &SchemaRef) -> Result<Dataset>
         enable_stable_row_ids: true,
         data_storage_version: Some(LanceFileVersion::V2_2),
         allow_external_blob_outside_bases: true,
+        auto_cleanup: None,
+        skip_auto_cleanup: true,
         ..Default::default()
     };
     Dataset::write(reader, uri, Some(params))

crates/omnigraph/src/db/manifest/namespace.rs

@@ -10,7 +10,9 @@ use lance_namespace::models::{
 };
 use lance_namespace::{Error as LanceNamespaceError, LanceNamespace, NamespaceError};
 use lance_table::io::commit::ManifestNamingScheme;
-use object_store::{Error as ObjectStoreError, ObjectStore as _, PutMode, PutOptions, path::Path};
+use object_store::{
+    Error as ObjectStoreError, ObjectStore as _, ObjectStoreExt, PutMode, PutOptions, path::Path,
+};
 
 use crate::error::{OmniError, Result};
 

crates/omnigraph/src/db/manifest/publisher.rs

@@ -381,6 +381,12 @@ impl GraphNamespacePublisher {
         // the publisher loop above, where each attempt re-runs the pre-check.
         merge_builder.conflict_retries(0);
         merge_builder.use_index(false);
+        // Skip Lance's auto-cleanup hook: `__manifest` versions are the snapshot
+        // / time-travel authority and must never be GC'd by Lance's per-commit
+        // hook. A `__manifest` created before the v7 bump (6.0.1 defaulted
+        // auto_cleanup ON) still carries the stored config, so this skip is
+        // load-bearing on upgraded graphs, not just defensive.
+        merge_builder.skip_auto_cleanup(true);
         let (new_dataset, _stats) = merge_builder
             .try_build()
             .map_err(|e| OmniError::Lance(e.to_string()))?

crates/omnigraph/src/db/recovery_audit.rs

@@ -189,6 +189,8 @@ async fn create_recoveries_dataset(root_uri: &str) -> Result<Dataset> {
         mode: WriteMode::Create,
         enable_stable_row_ids: true,
         data_storage_version: Some(LanceFileVersion::V2_2),
+        auto_cleanup: None,
+        skip_auto_cleanup: true,
         ..Default::default()
     };
     match Dataset::write(reader, &uri as &str, Some(params)).await {

crates/omnigraph/src/storage.rs

@@ -9,7 +9,7 @@ use object_store::aws::AmazonS3Builder;
 use object_store::local::LocalFileSystem;
 use object_store::memory::InMemory;
 use object_store::path::Path as ObjectPath;
-use object_store::{DynObjectStore, ObjectStore, PutMode, PutPayload};
+use object_store::{DynObjectStore, ObjectStore, ObjectStoreExt, PutMode, PutPayload};
 use url::Url;
 
 use crate::error::{OmniError, Result};

crates/omnigraph/src/table_store.rs

@@ -775,6 +775,8 @@ impl TableStore {
         let params = WriteParams {
             mode: WriteMode::Append,
             allow_external_blob_outside_bases: true,
+            auto_cleanup: None,
+            skip_auto_cleanup: true,
             ..Default::default()
         };
         ds.append(reader, Some(params))
@@ -794,6 +796,8 @@ impl TableStore {
                 let params = WriteParams {
                     mode: WriteMode::Append,
                     allow_external_blob_outside_bases: true,
+                    auto_cleanup: None,
+                    skip_auto_cleanup: true,
                     ..Default::default()
                 };
                 ds.append(reader, Some(params))
@@ -807,6 +811,8 @@ impl TableStore {
                     enable_stable_row_ids: true,
                     data_storage_version: Some(LanceFileVersion::V2_2),
                     allow_external_blob_outside_bases: true,
+                    auto_cleanup: None,
+                    skip_auto_cleanup: true,
                     ..Default::default()
                 };
                 Dataset::write(reader, dataset_uri, Some(params))
@@ -897,6 +903,8 @@ impl TableStore {
         let params = WriteParams {
             mode: WriteMode::Append,
             allow_external_blob_outside_bases: true,
+            auto_cleanup: None,
+            skip_auto_cleanup: true,
             ..Default::default()
         };
         let transaction = InsertBuilder::new(Arc::new(ds.clone()))
@@ -1070,7 +1078,16 @@ impl TableStore {
         ds: Arc<Dataset>,
         transaction: Transaction,
     ) -> Result<Dataset> {
+        // Skip Lance's auto-cleanup hook on every commit. OmniGraph owns version
+        // GC explicitly (optimize.rs::cleanup_all_tables); Lance's hook fires off
+        // the *dataset's stored* `lance.auto_cleanup.*` config, which graphs
+        // created before the v7 bump (6.0.1 defaulted auto_cleanup ON) still
+        // carry — so `WriteParams::auto_cleanup = None` alone does NOT stop it on
+        // upgraded graphs. Skipping here covers the staged write path (the main
+        // data path) for new and legacy datasets alike, preventing Lance from
+        // GC'ing versions the __manifest still pins for snapshots/time-travel.
         CommitBuilder::new(ds)
+            .with_skip_auto_cleanup(true)
             .execute(transaction)
             .await
             .map_err(|e| OmniError::Lance(e.to_string()))
@@ -1117,6 +1134,8 @@ impl TableStore {
                 mode: WriteMode::Overwrite,
                 enable_stable_row_ids: true,
                 allow_external_blob_outside_bases: true,
+                auto_cleanup: None,
+                skip_auto_cleanup: true,
                 ..Default::default()
             };
             let transaction = InsertBuilder::new(Arc::new(ds.clone()))
@@ -1533,6 +1552,8 @@ impl TableStore {
             enable_stable_row_ids: true,
             data_storage_version: Some(LanceFileVersion::V2_2),
             allow_external_blob_outside_bases: true,
+            auto_cleanup: None,
+            skip_auto_cleanup: true,
             ..Default::default()
         };
         Dataset::write(reader, dataset_uri, Some(params))

crates/omnigraph/tests/lance_surface_guards.rs

@@ -32,7 +32,10 @@ use lance::dataset::builder::DatasetBuilder;
 use lance::dataset::optimize::{CompactionOptions, compact_files};
 use lance::dataset::transaction::Operation;
 use lance::dataset::write::delete::DeleteResult;
-use lance::dataset::{MergeInsertBuilder, WhenMatched, WhenNotMatched, WriteMode, WriteParams};
+use lance::dataset::{
+    CommitBuilder, InsertBuilder, MergeInsertBuilder, WhenMatched, WhenNotMatched, WriteMode,
+    WriteParams,
+};
 use lance::index::DatasetIndexExt;
 use lance_file::version::LanceFileVersion;
 use lance_index::IndexType;
@@ -738,3 +741,180 @@ async fn scalar_index_use_requires_matched_literal_type() {
         "expected a column-side cast in the widened plan, got:\n{widened}"
     );
 }
+
+// --- Guard 17: BTREE scalar-index range-boundary correctness (lance#6796) -----
+//
+// lance#6796 (issue #6792) fixed a BTREE range-query bound-inclusiveness bug:
+// `price <= 10 AND price > 5` returned the wrong boundary row (5.0 instead of
+// 10.0). OmniGraph today builds BTREE only on string `@key` columns and queries
+// them by equality/IN, not range, so its current patterns do not hit this — the
+// guard protects any future BTREE-range path. It reproduces the exact #6792 shape
+// (5 rows + an explicit BTREE drives the index path even on tiny data, per the
+// upstream repro) and pins the corrected inclusive-`<=` / exclusive-`>` semantics.
+#[tokio::test]
+async fn btree_range_query_boundary_is_correct() {
+    use arrow_array::Float64Array;
+    use futures::TryStreamExt;
+
+    let dir = tempfile::tempdir().unwrap();
+    let uri = dir.path().join("guard17.lance");
+    let schema = Arc::new(Schema::new(vec![
+        Field::new("id", DataType::Utf8, false),
+        Field::new("price", DataType::Float64, false),
+    ]));
+    let batch = RecordBatch::try_new(
+        schema.clone(),
+        vec![
+            Arc::new(StringArray::from(vec!["a", "b", "c", "d", "e"])),
+            Arc::new(Float64Array::from(vec![1.0, 5.0, 10.0, 15.0, 20.0])),
+        ],
+    )
+    .unwrap();
+    let reader = RecordBatchIterator::new(vec![Ok(batch)], schema);
+    let params = WriteParams {
+        mode: WriteMode::Create,
+        enable_stable_row_ids: true,
+        data_storage_version: Some(LanceFileVersion::V2_2),
+        ..Default::default()
+    };
+    let mut ds = Dataset::write(reader, uri.to_str().unwrap(), Some(params))
+        .await
+        .unwrap();
+
+    // Build the BTREE on the numeric column so the range filter resolves through
+    // the scalar index (the path lance#6796 fixed).
+    ds.create_index_builder(&["price"], IndexType::BTree, &ScalarIndexParams::default())
+        .replace(true)
+        .await
+        .unwrap();
+
+    let mut scanner = ds.scan();
+    scanner.filter("price <= 10.0 AND price > 5.0").unwrap();
+    let batches: Vec<RecordBatch> = scanner
+        .try_into_stream()
+        .await
+        .unwrap()
+        .try_collect()
+        .await
+        .unwrap();
+    let mut got: Vec<f64> = Vec::new();
+    for b in &batches {
+        let col = b
+            .column_by_name("price")
+            .unwrap()
+            .as_any()
+            .downcast_ref::<Float64Array>()
+            .unwrap();
+        for i in 0..col.len() {
+            got.push(col.value(i));
+        }
+    }
+    got.sort_by(|a, b| a.partial_cmp(b).unwrap());
+    assert_eq!(
+        got,
+        vec![10.0],
+        "BTREE range `price <= 10 AND price > 5` must return exactly [10.0] \
+         (lance#6796 / issue #6792 boundary fix); got {got:?}. If this regressed, \
+         Lance reintroduced the range-bound inclusiveness bug.",
+    );
+}
+
+// --- Guard 18: skip_auto_cleanup suppresses version GC (lance#6755 / PR #229) --
+//
+// After the v7 bump, OmniGraph relies on `CommitBuilder::with_skip_auto_cleanup`
+// (`commit_staged`) and `MergeInsertBuilder::skip_auto_cleanup` (the `__manifest`
+// publisher) to stop Lance's per-commit auto-cleanup hook from GC'ing versions
+// the `__manifest` pins for snapshots/time-travel. This is load-bearing for
+// graphs created BEFORE the bump: 6.0.1 defaulted `WriteParams::auto_cleanup` ON,
+// so those datasets carry `lance.auto_cleanup.*` config that `auto_cleanup = None`
+// on new writes cannot retroactively clear — only the per-commit skip stops it.
+//
+// Pins both halves: WITHOUT the skip the aggressive config GCs v1; WITH the skip
+// (the exact call `commit_staged` makes) v1 survives.
+#[tokio::test]
+async fn skip_auto_cleanup_suppresses_version_gc() {
+    use std::collections::HashMap;
+
+    // The cleanup config 6.0.1 stored by default, made aggressive: fire on every
+    // commit, delete anything older than now.
+    async fn set_legacy_cleanup(ds: &mut Dataset) {
+        let mut cfg = HashMap::new();
+        cfg.insert("lance.auto_cleanup.interval".to_string(), "1".to_string());
+        cfg.insert("lance.auto_cleanup.older_than".to_string(), "0ms".to_string());
+        ds.update_config(cfg).await.unwrap();
+    }
+    fn row(i: i32) -> (Arc<Schema>, RecordBatch) {
+        let schema = Arc::new(Schema::new(vec![
+            Field::new("id", DataType::Utf8, false),
+            Field::new("value", DataType::Int32, false),
+        ]));
+        let batch = RecordBatch::try_new(
+            schema.clone(),
+            vec![
+                Arc::new(StringArray::from(vec![format!("k{i}")])),
+                Arc::new(Int32Array::from(vec![i])),
+            ],
+        )
+        .unwrap();
+        (schema, batch)
+    }
+
+    // Negative control: WITHOUT skip, the legacy config GCs the pinned v1.
+    let ctrl = tempfile::tempdir().unwrap();
+    let curi = ctrl.path().join("g18_ctrl.lance");
+    let curi = curi.to_str().unwrap();
+    let mut ds = fresh_dataset(curi).await;
+    let v1 = ds.version().version;
+    set_legacy_cleanup(&mut ds).await;
+    for i in 0..5 {
+        let (schema, batch) = row(i);
+        let reader = RecordBatchIterator::new(vec![Ok(batch)], schema);
+        ds.append(
+            reader,
+            Some(WriteParams {
+                mode: WriteMode::Append,
+                ..Default::default()
+            }),
+        )
+        .await
+        .unwrap();
+    }
+    assert!(
+        ds.checkout_version(v1).await.is_err(),
+        "negative control: without skip_auto_cleanup, the legacy auto_cleanup \
+         config should have GC'd pinned v{v1}; if this fails the config is not \
+         firing and the positive assertion below proves nothing."
+    );
+
+    // The guarantee: WITH the per-commit skip, v1 survives. Mirrors
+    // `TableStore::commit_staged` (InsertBuilder::execute_uncommitted +
+    // CommitBuilder::with_skip_auto_cleanup(true)).
+    let keep = tempfile::tempdir().unwrap();
+    let kuri = keep.path().join("g18.lance");
+    let kuri = kuri.to_str().unwrap();
+    let mut ds = fresh_dataset(kuri).await;
+    let v1 = ds.version().version;
+    set_legacy_cleanup(&mut ds).await;
+    for i in 0..5 {
+        let (_schema, batch) = row(i);
+        let tx = InsertBuilder::new(Arc::new(ds.clone()))
+            .with_params(&WriteParams {
+                mode: WriteMode::Append,
+                ..Default::default()
+            })
+            .execute_uncommitted(vec![batch])
+            .await
+            .unwrap();
+        ds = CommitBuilder::new(Arc::new(ds.clone()))
+            .with_skip_auto_cleanup(true)
+            .execute(tx)
+            .await
+            .unwrap();
+    }
+    assert!(
+        ds.checkout_version(v1).await.is_ok(),
+        "v{v1} was GC'd despite CommitBuilder::with_skip_auto_cleanup(true) — the \
+         commit_staged / publisher skip is the only thing protecting \
+         __manifest-pinned versions on upgraded (pre-bump) graphs."
+    );
+}

crates/omnigraph/tests/lance_version_columns.rs

@@ -191,14 +191,16 @@ async fn lance_merge_insert_new_row_stamps_created_at_version() {
     let eve = rows.iter().find(|r| r.0 == "eve").unwrap();
     eprintln!("Eve: created_at_version={}, v1={}, v2={}", eve.2, v1, v2);
 
-    // Lance behavior (as of 3.0.1): merge_insert stamps new rows with
-    // _row_created_at_version = dataset_creation_version (v1), NOT the
-    // merge_insert commit version (v2). This is why Omnigraph's change
-    // detection uses _row_last_updated_at_version + ID set membership
-    // to classify inserts vs updates, not _row_created_at_version alone.
+    // Lance behavior (7.0.0, lance#6774): merge_insert stamps new INSERT
+    // rows with _row_created_at_version = the commit version (v2). Earlier
+    // Lance used a fallback of the dataset creation version; #6774 changed
+    // it so created_at reflects when the row actually entered the dataset.
+    // Omnigraph's change detection keys on _row_last_updated_at_version + ID
+    // set membership (see changes/mod.rs), so this stamping change leaves
+    // insert-vs-update classification unaffected.
     assert_eq!(
-        eve.2, v1,
-        "Lance merge_insert stamps new rows with created_at = dataset creation version, not commit version"
+        eve.2, v2,
+        "Lance merge_insert stamps new rows with created_at = commit version (lance#6774)"
     );
     assert_eq!(
         eve.3, v2,
@@ -258,11 +260,24 @@ async fn lance_merge_insert_update_preserves_created_at_version() {
     assert_eq!(alice.2, v1, "alice created_at should still be v1");
     assert_eq!(alice.3, v1, "alice updated_at should still be v1");
 
-    // Bob: updated via merge_insert
-    // created_at should be preserved (v1), updated_at should be bumped (v2)
+    // Bob: updated via merge_insert.
     eprintln!(
         "Bob: created_at={}, updated_at={}, v1={}, v2={}",
         bob.2, bob.3, v1, v2
     );
     assert_eq!(bob.1, 99, "bob's value should be updated to 99");
+    // created_at is preserved across an UPDATE (lance#6774 only changed the
+    // INSERT-row stamping), which is what this test's name promises.
+    assert_eq!(
+        bob.2, v1,
+        "bob created_at must be preserved across a merge_insert UPDATE"
+    );
+    // updated_at bumps to the commit version on UPDATE — the change-feed
+    // invariant OmniGraph's insert/update classification relies on
+    // (changes/mod.rs keys on _row_last_updated_at_version). If this regresses,
+    // the diff/change feed silently misses updates.
+    assert_eq!(
+        bob.3, v2,
+        "bob updated_at must bump to the commit version on a merge_insert UPDATE"
+    );
 }

crates/omnigraph/tests/staged_writes.rs

@@ -1046,3 +1046,54 @@ async fn lance_restore_loses_to_concurrent_append_via_orphaning() {
     let v2_ids = collect_ids(&v2_batches);
     assert_eq!(v2_ids, vec!["alice".to_string(), "bob".to_string()]);
 }
+
+/// Regression for PR #229: `commit_staged` must skip Lance's per-commit
+/// auto-cleanup hook. A graph created BEFORE the v7 bump (6.0.1 defaulted
+/// `WriteParams::auto_cleanup` ON) carries `lance.auto_cleanup.*` config on its
+/// datasets that `auto_cleanup = None` on new writes cannot retroactively clear;
+/// Lance's hook fires off that *stored* config at commit time. Without the skip,
+/// the engine's own writes would GC the versions `__manifest` pins for
+/// snapshots/time-travel. (The substrate negative control — that the config
+/// really does GC without the skip — lives in
+/// `lance_surface_guards.rs::skip_auto_cleanup_suppresses_version_gc`.)
+#[tokio::test]
+async fn commit_staged_skips_auto_cleanup_so_pinned_versions_survive() {
+    use std::collections::HashMap;
+
+    let dir = tempfile::tempdir().unwrap();
+    let uri = format!("{}/people.lance", dir.path().to_str().unwrap());
+    let store = TableStore::new(dir.path().to_str().unwrap());
+
+    let mut ds = TableStore::write_dataset(&uri, person_batch(&[("seed", Some(0))]))
+        .await
+        .unwrap();
+    let v1 = ds.version().version;
+
+    // Simulate a pre-bump dataset: aggressive legacy auto_cleanup config (fire on
+    // every commit, delete anything older than now).
+    let mut cfg = HashMap::new();
+    cfg.insert("lance.auto_cleanup.interval".to_string(), "1".to_string());
+    cfg.insert("lance.auto_cleanup.older_than".to_string(), "0ms".to_string());
+    ds.update_config(cfg).await.unwrap();
+
+    // Several writes through the engine's staged commit path.
+    for i in 0..5i32 {
+        let name = format!("p{i}");
+        let staged = store
+            .stage_append(&ds, person_batch(&[(name.as_str(), Some(i))]), &[])
+            .await
+            .unwrap();
+        ds = store
+            .commit_staged(Arc::new(ds.clone()), staged.transaction)
+            .await
+            .unwrap();
+    }
+
+    // `commit_staged` sets `with_skip_auto_cleanup(true)`, so the legacy config
+    // must NOT have GC'd the `__manifest`-pinned create version.
+    assert!(
+        ds.checkout_version(v1).await.is_ok(),
+        "commit_staged must skip Lance auto-cleanup so a pre-bump graph's pinned \
+         v{v1} survives; it was GC'd"
+    );
+}