mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-06-12 01:45:14 +02:00
MR-794 step 2: address PR #68 follow-up review (Cubic) — pending dedupe + projection guard + CI
Three new findings from Cubic on commit3223b51: * **Pending edge cardinality counted within-input duplicates** (P2): count_src_per_edge's pending walk added every row to the count, including duplicate rows that finalize will collapse via dedupe_merge_batches_by_id. A LoadMode::Merge with the same edge id twice would over-count → spurious @card violation. Fix: when dedupe_key_column is Some, walk pending in reverse, track seen keys via HashSet, count only the kept (last-occurrence) rows. Mirrors finalize-time dedupe so cardinality counts what stage_merge_insert actually publishes. * **scan_with_pending silently disabled merge-shadow when projection omitted key_column** (P2): if a caller passed Some("id") as key_column but their projection didn't include "id", the filter_out_rows_where_string_in helper passed batches through unchanged — silently degrading to union semantics. Fix: validate up front that projection contains key_column when both are Some; return a typed Lance error otherwise. Tightened the helper too: missing column is now an internal error (was a silent passthrough). * **Cascade-vs-explicit delete test was too weak** (P2): asserted only that edge count decreased after delete. The cascade alone could satisfy that even if the explicit second-delete silently no-op'd. Strengthened: assert post_knows == 0, which only holds when both ops landed (Bob→Diana would survive if op-2 no-op'd). CI gap: also added test_failpoints_feature job to .github/workflows/ci.yml. The workspace test runs without --features failpoints (the feature is behind a Cargo flag), so the failpoints test suite was never exercised by CI before now. The new job builds + runs `cargo test -p omnigraph-engine --features failpoints --test failpoints` on every full CI run, mirroring the test_aws_feature pattern. New tests on tests/runs.rs: * load_merge_mode_dedupes_within_pending_for_cardinality_count (Cubic P2 #2 — pending-vs-pending dedup, distinct from the load_merge_mode_dedupes_edge_for_cardinality_count test which covers committed-vs-pending dedup). * scan_with_pending_rejects_key_column_missing_from_projection (Cubic P2 #3 — verifies the up-front validation rejects bad callers and that the happy path still works correctly). Local test results: * tests/runs.rs: 23/23 passed * tests/failpoints.rs --features failpoints: 7/7 passed (includes the two new finalize→publisher residual tests landed in3223b51). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
3223b51cf1
commit
052b6e680f
4 changed files with 306 additions and 16 deletions
42
.github/workflows/ci.yml
vendored
42
.github/workflows/ci.yml
vendored
|
|
@ -239,6 +239,48 @@ jobs:
|
|||
if: needs.classify_changes.outputs.run_full_ci == 'true'
|
||||
run: cargo test --locked -p omnigraph-server --features aws
|
||||
|
||||
test_failpoints_feature:
|
||||
name: Test omnigraph-engine --features failpoints
|
||||
needs: classify_changes
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
CARGO_TERM_COLOR: always
|
||||
steps:
|
||||
- name: Skip for text-only changes
|
||||
if: needs.classify_changes.outputs.run_full_ci != 'true'
|
||||
run: echo "Text-only change detected; skipping failpoints feature build."
|
||||
|
||||
- name: Checkout source
|
||||
if: needs.classify_changes.outputs.run_full_ci == 'true'
|
||||
uses: actions/checkout@v5.0.1
|
||||
|
||||
- name: Install system dependencies
|
||||
if: needs.classify_changes.outputs.run_full_ci == 'true'
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y protobuf-compiler libprotobuf-dev
|
||||
|
||||
- name: Install Rust stable
|
||||
if: needs.classify_changes.outputs.run_full_ci == 'true'
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
with:
|
||||
toolchain: stable
|
||||
|
||||
- name: Cache Rust build data
|
||||
if: needs.classify_changes.outputs.run_full_ci == 'true'
|
||||
uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
workspaces: |
|
||||
. -> target
|
||||
key: failpoints-feature
|
||||
|
||||
- name: Run failpoints test suite
|
||||
if: needs.classify_changes.outputs.run_full_ci == 'true'
|
||||
run: cargo test --locked -p omnigraph-engine --features failpoints --test failpoints
|
||||
|
||||
rustfs_integration:
|
||||
name: RustFS S3 Integration
|
||||
needs:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue