mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-06-09 01:35:18 +02:00
docs: rename runs.md/runs.rs → writes and repoint all references (#131)
Some checks failed
CI / Classify Changes (push) Has been cancelled
CI / Check AGENTS.md Links (push) Has been cancelled
CI / Container Entrypoint (push) Has been cancelled
Release Edge / Prepare edge release (push) Has been cancelled
CI / Test Workspace (push) Has been cancelled
CI / Test omnigraph-server --features aws (push) Has been cancelled
CI / Test Windows release binaries (push) Has been cancelled
CI / RustFS S3 Integration (push) Has been cancelled
Release Edge / Build edge omnigraph-linux-x86_64 (push) Has been cancelled
Release Edge / Build edge omnigraph-macos-arm64 (push) Has been cancelled
Release Edge / Build edge omnigraph-windows-x86_64 (push) Has been cancelled
Release Edge / Smoke Windows installer (push) Has been cancelled
Some checks failed
CI / Classify Changes (push) Has been cancelled
CI / Check AGENTS.md Links (push) Has been cancelled
CI / Container Entrypoint (push) Has been cancelled
Release Edge / Prepare edge release (push) Has been cancelled
CI / Test Workspace (push) Has been cancelled
CI / Test omnigraph-server --features aws (push) Has been cancelled
CI / Test Windows release binaries (push) Has been cancelled
CI / RustFS S3 Integration (push) Has been cancelled
Release Edge / Build edge omnigraph-linux-x86_64 (push) Has been cancelled
Release Edge / Build edge omnigraph-macos-arm64 (push) Has been cancelled
Release Edge / Build edge omnigraph-windows-x86_64 (push) Has been cancelled
Release Edge / Smoke Windows installer (push) Has been cancelled
The Run state machine was removed in MR-771 (v0.4.0); `docs/dev/runs.md` and `crates/omnigraph/tests/runs.rs` have since documented and tested the direct-publish write path, so the "runs" name was misleading. - git mv docs/dev/runs.md → docs/dev/writes.md (reframe H1 + intro; keep MR-771 history note) - git mv crates/omnigraph/tests/runs.rs → tests/writes.rs (reframe header) - repoint every runs.md / runs.rs reference across docs, AGENTS.md, and source comments - fix four pre-existing broken `docs/runs.md` links (the file never lived at that path) to `docs/dev/writes.md` - fix the stale v0.4.0 anchor to the live section No behavior change: every source edit is a comment. Engine builds and the renamed test passes 25/25; scripts/check-agents-md.sh passes. The run-removal cleanup itself (run_registry.rs guard, __run__ prefix) is deferred to MR-770.
This commit is contained in:
parent
854ad0afcb
commit
2d5c4b1202
18 changed files with 45 additions and 42 deletions
|
|
@ -207,7 +207,7 @@ contracts:
|
|||
This pattern realizes read-your-writes within a multi-statement mutation
|
||||
and keeps failure scope bounded for inserts/updates by construction at
|
||||
the writer layer. See [docs/dev/invariants.md](invariants.md) and
|
||||
[docs/dev/runs.md](runs.md) for the publisher CAS contract this builds on.
|
||||
[docs/dev/writes.md](writes.md) for the publisher CAS contract this builds on.
|
||||
|
||||
### Storage trait — today vs. roadmap
|
||||
|
||||
|
|
@ -278,7 +278,7 @@ flowchart LR
|
|||
eng --> wq
|
||||
```
|
||||
|
||||
The server applies Cedar policy at the HTTP boundary today. The roadmap, called out in [docs/dev/invariants.md](invariants.md) as a known gap, is to push policy into the planner as predicates. After Cedar, mutating handlers go through `WorkloadController` (per-actor admission cap + byte budget; PR 2 / MR-686) before reaching the engine. The engine itself holds an `Arc<WriteQueueManager>` so concurrent mutations on the same `(table, branch)` serialize at the queue, while disjoint keys run in parallel — see [docs/user/server.md](../user/server.md) "Per-actor admission control" and [docs/dev/runs.md](runs.md). The CLI bypasses the HTTP layer (and admission) and calls the engine API directly.
|
||||
The server applies Cedar policy at the HTTP boundary today. The roadmap, called out in [docs/dev/invariants.md](invariants.md) as a known gap, is to push policy into the planner as predicates. After Cedar, mutating handlers go through `WorkloadController` (per-actor admission cap + byte budget; PR 2 / MR-686) before reaching the engine. The engine itself holds an `Arc<WriteQueueManager>` so concurrent mutations on the same `(table, branch)` serialize at the queue, while disjoint keys run in parallel — see [docs/user/server.md](../user/server.md) "Per-actor admission control" and [docs/dev/writes.md](writes.md). The CLI bypasses the HTTP layer (and admission) and calls the engine API directly.
|
||||
|
||||
Code paths:
|
||||
|
||||
|
|
|
|||
|
|
@ -147,7 +147,7 @@ sequenceDiagram
|
|||
- End-of-query Lance commit: `TableStore::stage_append`, `stage_merge_insert`, `commit_staged` at `crates/omnigraph/src/table_store.rs`
|
||||
- Manifest commit primitive: `commit_updates_on_branch_with_expected` at `crates/omnigraph/src/db/omnigraph/table_ops.rs`
|
||||
|
||||
Atomicity guarantee for multi-statement mutations: a mid-query failure leaves Lance HEAD untouched on staged tables (no inline commit happened during op execution), so the next mutation proceeds normally with no `ExpectedVersionMismatch`. The publisher CAS at the very end either succeeds (manifest advances atomically across all touched sub-tables) or fails with a typed `ManifestConflictDetails::ExpectedVersionMismatch` (no partial publish). See [docs/dev/invariants.md](invariants.md) and [docs/dev/runs.md](runs.md).
|
||||
Atomicity guarantee for multi-statement mutations: a mid-query failure leaves Lance HEAD untouched on staged tables (no inline commit happened during op execution), so the next mutation proceeds normally with no `ExpectedVersionMismatch`. The publisher CAS at the very end either succeeds (manifest advances atomically across all touched sub-tables) or fails with a typed `ManifestConflictDetails::ExpectedVersionMismatch` (no partial publish). See [docs/dev/invariants.md](invariants.md) and [docs/dev/writes.md](writes.md).
|
||||
|
||||
## Bulk loader (`loader/mod.rs`)
|
||||
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ constraints. User-facing behavior should still be documented through
|
|||
|---|---|
|
||||
| System structure, L1/L2 framing, component diagrams | [architecture.md](architecture.md) |
|
||||
| On-disk layout, manifest schema, URI behavior | [storage.md](../user/storage.md) |
|
||||
| Direct-publish writes, D2, staged writes, recovery sidecars | [runs.md](runs.md) |
|
||||
| Direct-publish writes, D2, staged writes, recovery sidecars | [writes.md](writes.md) |
|
||||
| Query execution, mutation execution, loader flow | [execution.md](execution.md) |
|
||||
| Index lifecycle and graph topology indexes | [indexes.md](../user/indexes.md) |
|
||||
| Branch and commit internals | [branches-commits.md](../user/branches-commits.md) |
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@ Use it this way:
|
|||
publishes one manifest update. Do not commit per statement. Delete-only
|
||||
queries are the documented inline residual; the parse-time D2 rule prevents
|
||||
mixing deletes with insert/update until Lance exposes two-phase delete.
|
||||
Read [runs.md](runs.md) and [execution.md](execution.md).
|
||||
Read [writes.md](writes.md) and [execution.md](execution.md).
|
||||
|
||||
5. **Recovery is part of the commit protocol.** Writers that can advance Lance
|
||||
HEAD before manifest publish must write `__recovery/{ulid}.json` sidecars.
|
||||
|
|
@ -56,7 +56,7 @@ Use it this way:
|
|||
branch they read even when index coverage is partial. Expensive index work
|
||||
should converge from manifest state instead of extending the critical write
|
||||
path. Scalar staged index builds and vector inline residuals are documented
|
||||
in [runs.md](runs.md) and [indexes.md](../user/indexes.md).
|
||||
in [writes.md](writes.md) and [indexes.md](../user/indexes.md).
|
||||
|
||||
8. **Schema identity survives renames.** Accepted schema identity must remain
|
||||
stable across type and property renames. Rename support belongs in migration
|
||||
|
|
@ -96,12 +96,12 @@ Use it this way:
|
|||
|
||||
| Area | Current state | Source |
|
||||
|---|---|---|
|
||||
| Multi-table commit | Manifest CAS plus recovery sidecars; not a single Lance primitive | [runs.md](runs.md), [architecture.md](architecture.md) |
|
||||
| Constructive mutations | In-memory `MutationStaging`, one end-of-query table commit per touched table, then one manifest publish | [runs.md](runs.md), [execution.md](execution.md) |
|
||||
| Deletes | Inline-commit residual; delete-only queries allowed, mixed insert/update/delete rejected by D2 | [query-language.md](../user/query-language.md), [runs.md](runs.md) |
|
||||
| Multi-table commit | Manifest CAS plus recovery sidecars; not a single Lance primitive | [writes.md](writes.md), [architecture.md](architecture.md) |
|
||||
| Constructive mutations | In-memory `MutationStaging`, one end-of-query table commit per touched table, then one manifest publish | [writes.md](writes.md), [execution.md](execution.md) |
|
||||
| Deletes | Inline-commit residual; delete-only queries allowed, mixed insert/update/delete rejected by D2 | [query-language.md](../user/query-language.md), [writes.md](writes.md) |
|
||||
| Schema validation | Type checks, required fields, defaults, edge endpoint checks, and edge cardinality are enforced on write paths | [schema-language.md](../user/schema-language.md), [execution.md](execution.md) |
|
||||
| Unique constraints | Intra-batch and write-path checks exist; full cross-version uniqueness is still a gap | [schema-language.md](../user/schema-language.md) |
|
||||
| Storage trait | `TableStorage` exists as the sealed staged-write surface; full call-site migration and capability/stat surfaces are incomplete | [runs.md](runs.md), [architecture.md](architecture.md) |
|
||||
| Storage trait | `TableStorage` exists as the sealed staged-write surface; full call-site migration and capability/stat surfaces are incomplete | [writes.md](writes.md), [architecture.md](architecture.md) |
|
||||
| Index lifecycle | `ensure_indices` is explicit today; reconciler-based convergence is roadmap | [indexes.md](../user/indexes.md), [maintenance.md](../user/maintenance.md) |
|
||||
| Traversal IDs | Runtime still builds `TypeIndex`; Lance stable row-id based graph IDs are roadmap | [architecture.md](architecture.md), [query-language.md](../user/query-language.md) |
|
||||
| Auth | Bearer token hashing and server-side actor resolution are implemented at the HTTP boundary | [server.md](../user/server.md), [policy.md](../user/policy.md) |
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav
|
|||
| `end_to_end.rs` | Full init → load → query/mutate flow |
|
||||
| `branching.rs` | Branch create / list / delete, lazy fork |
|
||||
| `merge_truth_table.rs` | Merge-pair truth table (MR-786): all 9×9 `(left_op, right_op)` cells from `{noop, addNode, removeNode, addEdge, removeEdge, setProperty, dropProperty, addLabel, removeLabel}`. Adding a new op to `OpVariant` forces a compile error in `build_case` until the new row + column are dispositioned. 36 executable cells run through real `branch_merge` with a structured oracle (`MergeOutcome` / `MergeConflictKind` + graph-state assert); 45 cells involving `dropProperty`/`addLabel`/`removeLabel` are recorded as `Unsupported` until the mutation grammar grows. |
|
||||
| `runs.rs` | Direct-publish writes: cancellation, concurrent-writer CAS, multi-statement atomicity, MR-794 staged-write rewire (D₂ rejection, insert+update coalesce, multi-append coalesce, partial-failure recovery, load RI/cardinality recovery) |
|
||||
| `writes.rs` | Direct-publish writes: cancellation, concurrent-writer CAS, multi-statement atomicity, MR-794 staged-write rewire (D₂ rejection, insert+update coalesce, multi-append coalesce, partial-failure recovery, load RI/cardinality recovery) |
|
||||
| `staged_writes.rs` | TableStore staged-write primitives (`stage_append`, `stage_merge_insert`, `commit_staged`, `scan_with_staged`, `count_rows_with_staged`) — primitive-level only; engine code uses the in-memory `MutationStaging` accumulator instead |
|
||||
| `lifecycle.rs` | Graph lifecycle, schema state |
|
||||
| `point_in_time.rs` | Snapshots, time travel (`snapshot_at_version`, `entity_at`) |
|
||||
|
|
@ -89,7 +89,7 @@ If introducing coverage tooling is in scope for your task, the natural first ste
|
|||
|
||||
How to check:
|
||||
|
||||
1. **Map the change to an area** — use the engine integration-test table above (`branching.rs`, `runs.rs`, `search.rs`, etc.). The filename usually names the area.
|
||||
1. **Map the change to an area** — use the engine integration-test table above (`branching.rs`, `writes.rs`, `search.rs`, etc.). The filename usually names the area.
|
||||
2. **Open the file and skim every test fn name.** Test fn names are the index — read them all, not just the first few.
|
||||
3. **Grep for the symbol or path you're changing.** `rg <FunctionName>` or `rg <enum_variant>` across all `tests/` directories surfaces existing coverage you might miss.
|
||||
4. **Decide one of three outcomes**, in this order of preference:
|
||||
|
|
|
|||
|
|
@ -1,7 +1,10 @@
|
|||
# Runs — REMOVED (MR-771)
|
||||
# Direct-Publish Write Path
|
||||
|
||||
The Run state machine and `__run__<id>` staging branches were removed in
|
||||
MR-771. `mutate_as` and `load` now write **directly to the target table**
|
||||
> History: the Run state machine and `__run__<id>` staging branches were
|
||||
> removed in MR-771 (shipped v0.4.0). Writes now go directly to the target
|
||||
> table; this document specifies that direct-publish path.
|
||||
|
||||
`mutate_as` and `load` write **directly to the target table**
|
||||
and call `ManifestBatchPublisher::publish` once at the end with
|
||||
`expected_table_versions` (the per-table manifest versions captured before
|
||||
the first write). Cross-table OCC is enforced inside the publisher; the
|
||||
|
|
@ -65,7 +65,7 @@ manifest. The next mutation against that table fails with
|
|||
`ExpectedVersionMismatch`. Most validation runs before any Lance write,
|
||||
so single-statement mutations are unaffected; the narrow path is
|
||||
multi-statement queries with late-op failures. Tracked as a follow-up;
|
||||
see [docs/dev/runs.md](../dev/runs.md#known-limitation-mid-query-partial-failure-on-the-same-table)
|
||||
see [docs/dev/writes.md](../dev/writes.md#mid-query-partial-failure-closed-by-mr-794)
|
||||
for the workaround.
|
||||
|
||||
## Upgrade notes
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ mutation proceeds normally.
|
|||
HEAD on every staged table is untouched and the next mutation
|
||||
proceeds normally. A narrowed residual remains at the
|
||||
finalize→publisher boundary (multi-table `commit_staged` is not
|
||||
atomic with the manifest commit) — see [docs/dev/runs.md](../dev/runs.md)
|
||||
atomic with the manifest commit) — see [docs/dev/writes.md](../dev/writes.md)
|
||||
"Finalize → publisher residual" for details.
|
||||
- **D₂ parse-time rule**: a single mutation query is either
|
||||
insert/update-only or delete-only. Mixed → rejected with a clear
|
||||
|
|
@ -75,14 +75,14 @@ mutation proceeds normally.
|
|||
|
||||
## Tests added
|
||||
|
||||
- `tests/runs.rs::partial_failure_leaves_target_queryable_and_unblocks_next_mutation`
|
||||
- `tests/writes.rs::partial_failure_leaves_target_queryable_and_unblocks_next_mutation`
|
||||
(replaces the old `partial_failure_observably_rolls_back_but_blocks_next_mutation_on_same_table`)
|
||||
- `tests/runs.rs::mutation_rejects_mixed_insert_and_delete_at_parse_time`
|
||||
- `tests/runs.rs::mixed_insert_and_update_on_same_person_coalesces_to_one_merge`
|
||||
- `tests/runs.rs::multiple_appends_to_same_edge_coalesce_to_one_append`
|
||||
- `tests/runs.rs::multi_statement_inserts_publish_exactly_once`
|
||||
- `tests/runs.rs::load_with_bad_edge_reference_unblocks_next_load`
|
||||
- `tests/runs.rs::load_with_cardinality_violation_unblocks_next_load`
|
||||
- `tests/writes.rs::mutation_rejects_mixed_insert_and_delete_at_parse_time`
|
||||
- `tests/writes.rs::mixed_insert_and_update_on_same_person_coalesces_to_one_merge`
|
||||
- `tests/writes.rs::multiple_appends_to_same_edge_coalesce_to_one_append`
|
||||
- `tests/writes.rs::multi_statement_inserts_publish_exactly_once`
|
||||
- `tests/writes.rs::load_with_bad_edge_reference_unblocks_next_load`
|
||||
- `tests/writes.rs::load_with_cardinality_violation_unblocks_next_load`
|
||||
|
||||
## Files changed
|
||||
|
||||
|
|
@ -105,7 +105,7 @@ mutation proceeds normally.
|
|||
- `Cargo.toml` (workspace) + `crates/omnigraph/Cargo.toml` — added
|
||||
`datafusion = "52"` direct dep (transitively pulled by Lance
|
||||
already; required for `MemTable`).
|
||||
- `docs/dev/runs.md` — removed "Known limitation" section; documented
|
||||
- `docs/dev/writes.md` — removed "Known limitation" section; documented
|
||||
the new accumulator + D₂ + LoadMode::Overwrite residual.
|
||||
- `docs/dev/invariants.md` — mutation atomicity / read-your-writes status
|
||||
flipped to `upheld for inserts/updates`.
|
||||
|
|
@ -127,7 +127,7 @@ mutation proceeds normally.
|
|||
as legacy.
|
||||
- `docs/user/cli.md` — replaced the legacy `omnigraph run *` quickstart
|
||||
block with `omnigraph commit list/show`.
|
||||
- `docs/dev/testing.md` — extended the `runs.rs` row to cover the new
|
||||
- `docs/dev/testing.md` — extended the `writes.rs` row to cover the new
|
||||
staged-write contract tests; added the `staged_writes.rs` row.
|
||||
- `AGENTS.md` (CLAUDE.md symlink) — updated the atomic-per-query
|
||||
description and the L2 capability matrix row.
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
- `Manifest(ManifestError { kind: BadRequest|NotFound|Conflict|Internal, details: Option<ManifestConflictDetails>, … })`
|
||||
- `ManifestConflictDetails::ExpectedVersionMismatch { table_key, expected, actual }` — caller's `expected_table_versions` did not match the manifest's current latest non-tombstoned version (set by `OmniError::manifest_expected_version_mismatch`).
|
||||
- `ManifestConflictDetails::RowLevelCasContention` — Lance row-level CAS rejected the publish because a concurrent writer landed the same `object_id`. Retried internally by the publisher; only surfaces if the retry budget exhausts.
|
||||
- **D₂ parse-time rejection** (MR-794): a single mutation query that mixes inserts/updates with deletes errors out *before any I/O* with kind `BadRequest`. Message: `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes`. See [docs/user/query-language.md](query-language.md) for the rule and [docs/dev/runs.md](../dev/runs.md) for the underlying staged-write rationale.
|
||||
- **D₂ parse-time rejection** (MR-794): a single mutation query that mixes inserts/updates with deletes errors out *before any I/O* with kind `BadRequest`. Message: `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes`. See [docs/user/query-language.md](query-language.md) for the rule and [docs/dev/writes.md](../dev/writes.md) for the underlying staged-write rationale.
|
||||
- `MergeConflicts(Vec<MergeConflict>)`
|
||||
|
||||
Compiler-side `NanoError` covers parse / catalog / type / storage / plan / execution / arrow / lance / IO / manifest / unique-constraint, each with structured spans (`SourceSpan { start, end }`) for ariadne-style diagnostics.
|
||||
|
|
|
|||
|
|
@ -70,7 +70,7 @@ A single mutation query must be **either insert/update-only or delete-only**. Mi
|
|||
|
||||
> `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes. This restriction lifts when Lance exposes a two-phase delete API (tracked: MR-793 / Lance-upstream).`
|
||||
|
||||
Reason: under the staged-write rewire (MR-794), inserts and updates accumulate in memory and commit at end-of-query, while deletes still inline-commit (Lance 4.0.0 has no public two-phase delete). Mixing creates ordering hazards (same-row insert→delete becomes a no-op because the staged insert isn't visible to delete; cascading deletes of just-inserted edges break referential integrity by silent design). Until Lance exposes `DeleteJob::execute_uncommitted`, the parse-time rejection keeps both paths atomic and correct. See [docs/dev/runs.md](../dev/runs.md) and [docs/dev/invariants.md](../dev/invariants.md).
|
||||
Reason: under the staged-write rewire (MR-794), inserts and updates accumulate in memory and commit at end-of-query, while deletes still inline-commit (Lance 4.0.0 has no public two-phase delete). Mixing creates ordering hazards (same-row insert→delete becomes a no-op because the staged insert isn't visible to delete; cascading deletes of just-inserted edges break referential integrity by silent design). Until Lance exposes `DeleteJob::execute_uncommitted`, the parse-time rejection keeps both paths atomic and correct. See [docs/dev/writes.md](../dev/writes.md) and [docs/dev/invariants.md](../dev/invariants.md).
|
||||
|
||||
## IR (Intermediate Representation)
|
||||
|
||||
|
|
|
|||
|
|
@ -164,5 +164,5 @@ This is the workflow MR-797 / agentic loops are designed around: **branches are
|
|||
- [`docs/user/branches-commits.md`](branches-commits.md) — branch and commit-graph mechanics.
|
||||
- [`docs/dev/merge.md`](../dev/merge.md) — three-way merge details and conflict kinds.
|
||||
- [`docs/user/query-language.md`](query-language.md) — `.gq` syntax for the multi-statement queries used above.
|
||||
- [`docs/dev/runs.md`](../dev/runs.md) — the per-query commit pipeline that gives single-query atomicity.
|
||||
- [`docs/dev/writes.md`](../dev/writes.md) — the per-query commit pipeline that gives single-query atomicity.
|
||||
- [`docs/dev/invariants.md`](../dev/invariants.md) — the architectural rule.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue