mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-07-15 03:22:11 +02:00
docs: audit pass — drop pre-0.7.0 release notes; scrub RFC refs from user docs (#272)
* docs: audit pass — drop pre-0.7.0 release notes; scrub RFC refs from user docs
- Delete the pre-0.7.0 release-notes archive (v0.2.0 … v0.6.2); keep v0.7.0.
- Rewrite every inline "RFC-0NN" citation in docs/user/** into durable
plain language (the behavior is the contract, not the planning doc):
cli/index.md, cli/reference.md, clusters/index.md, operations/{maintenance,
policy,server}.md. Updated the in-page "Scopes & profiles" anchor to match
the de-RFC'd heading.
No sub-0.7.0 version caveats or stale Lance-version refs were present in
docs/user/**. Dev docs, AGENTS.md, and instruction files are out of scope for
this pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: second alignment pass — drop residual pre-cluster-only framing
- cli/reference.md: rewrite the server-scope graph-resolution rule — an
omnigraph-server is always cluster-backed, so GET /graphs always answers and
--graph is required; the bare-URL path is only the fallback for an
unavailable/non-omnigraph endpoint (was "a single-graph / flat server …
uses its bare URL as before").
- embeddings.md: "Direct single-graph serving" → "Direct (--store) access"
(there is no single-graph serving mode under cluster-only).
- clusters/{config,index}.md: drop the removed --target flag from the
"--cluster cannot combine with …" clauses.
Verified: no Linear tickets, no RFC refs, no single-graph-as-current, no
--target-as-combinable in docs/user/**.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
b6131393b7
commit
d0e06a6ff6
20 changed files with 23 additions and 1018 deletions
|
|
@ -1,142 +0,0 @@
|
|||
# Omnigraph v0.4.1
|
||||
|
||||
Omnigraph v0.4.1 closes the multi-statement-mutation atomicity gap that
|
||||
v0.4.0 documented as a known limitation. Inserts and updates now route
|
||||
through an in-memory `MutationStaging` accumulator and commit via Lance's
|
||||
two-phase distributed-write API at end-of-query. A failed mid-query op
|
||||
no longer leaves Lance HEAD drifted on the touched table — the next
|
||||
mutation proceeds normally.
|
||||
|
||||
## Highlights
|
||||
|
||||
- **Staged-write rewire**: `mutate_as` and `load` (Append /
|
||||
Merge modes) accumulate insert/update batches into
|
||||
`MutationStaging.pending` per touched table. No Lance HEAD advance
|
||||
happens during op execution; one `stage_*` + `commit_staged` per
|
||||
table runs at end-of-query, then `ManifestBatchPublisher::publish`
|
||||
commits the manifest atomically. **For op-execution failures**
|
||||
(validation errors, missing endpoints, parse-time D₂ rejection), Lance
|
||||
HEAD on every staged table is untouched and the next mutation
|
||||
proceeds normally. A narrowed residual remains at the
|
||||
finalize→publisher boundary (multi-table `commit_staged` is not
|
||||
atomic with the manifest commit) — see [docs/dev/writes.md](../dev/writes.md)
|
||||
"Finalize → publisher residual" for details.
|
||||
- **D₂ parse-time rule**: a single mutation query is either
|
||||
insert/update-only or delete-only. Mixed → rejected with a clear
|
||||
error directing the caller to split into two queries. Lance 4.0.0
|
||||
has no public two-phase delete; deletes still inline-commit, and D₂
|
||||
keeps that path safe.
|
||||
- **Read-your-writes via DataFusion `MemTable`**: read sites in
|
||||
multi-statement mutations consume `TableStore::scan_with_pending`,
|
||||
which Lance-scans the committed snapshot at the captured
|
||||
`expected_version` and unions with a DataFusion `MemTable` over the
|
||||
pending batches. Replaces the previous "reopen at staged Lance
|
||||
version" pattern.
|
||||
- **Coordinator swap-restore eliminated** from `mutate_with_current_actor`.
|
||||
Branch is threaded explicitly through the per-op execution path
|
||||
(`execute_named_mutation`, `execute_insert`, `execute_update`,
|
||||
`execute_delete*`, `validate_edge_insert_endpoints`,
|
||||
`ensure_node_id_exists`). The `swap_coordinator_for_branch` /
|
||||
`restore_coordinator` API and `CoordinatorRestoreGuard` are removed
|
||||
from `mutation.rs`. (`merge.rs` keeps its own swap pattern; that's
|
||||
a separate workflow.)
|
||||
- **`docs/dev/invariants.md` mutation atomicity / read-your-writes status**
|
||||
flips from `aspirational/open` to `upheld for inserts/updates`. The within-query read-your-writes
|
||||
guarantee is now load-bearing for the publisher CAS contract.
|
||||
|
||||
## Behavior changes
|
||||
|
||||
- A failed multi-statement mutation no longer surfaces
|
||||
`ExpectedVersionMismatch` on the *next* mutation against the same
|
||||
table. The next call proceeds normally — Lance HEAD on staged
|
||||
tables is unchanged.
|
||||
- Mixed insert/update + delete in one query is rejected at parse
|
||||
time. Existing test queries that mixed both must be split.
|
||||
- `MutationStaging`'s shape changed: `pending: HashMap<String, PendingTable>`
|
||||
+ `inline_committed: HashMap<String, SubTableUpdate>` replaces the
|
||||
previous `latest: HashMap<String, StagedTable>`. This is an internal
|
||||
type; no public API impact.
|
||||
|
||||
## Residual / out of scope
|
||||
|
||||
- **`LoadMode::Overwrite`** keeps the legacy inline-commit path
|
||||
(truncate-then-append doesn't fit the staged shape). A mid-overwrite
|
||||
failure can still drift Lance HEAD on a partially-truncated table;
|
||||
the next overwrite replaces it. Operator-driven, rare.
|
||||
- **Delete-only multi-statement mutations** still inline-commit per op.
|
||||
D₂ keeps inserts/updates from coexisting with deletes, so the
|
||||
inline path remains atomic per op but not per query for delete-only
|
||||
cascades. Closing this requires Lance to expose
|
||||
`DeleteJob::execute_uncommitted`; tracked upstream with Lance.
|
||||
- **`schema_apply`, `branch_merge_internal`, `ensure_indices`** still
|
||||
use Lance's inline-commit APIs. The two-phase pattern is in
|
||||
`mutate_as` and `load` only; hoisting it to a storage-trait invariant
|
||||
covering all writers remains future work.
|
||||
|
||||
## Tests added
|
||||
|
||||
- `tests/writes.rs::partial_failure_leaves_target_queryable_and_unblocks_next_mutation`
|
||||
(replaces the old `partial_failure_observably_rolls_back_but_blocks_next_mutation_on_same_table`)
|
||||
- `tests/writes.rs::mutation_rejects_mixed_insert_and_delete_at_parse_time`
|
||||
- `tests/writes.rs::mixed_insert_and_update_on_same_person_coalesces_to_one_merge`
|
||||
- `tests/writes.rs::multiple_appends_to_same_edge_coalesce_to_one_append`
|
||||
- `tests/writes.rs::multi_statement_inserts_publish_exactly_once`
|
||||
- `tests/writes.rs::load_with_bad_edge_reference_unblocks_next_load`
|
||||
- `tests/writes.rs::load_with_cardinality_violation_unblocks_next_load`
|
||||
|
||||
## Files changed
|
||||
|
||||
- `crates/omnigraph/src/exec/staging.rs` (NEW) — `MutationStaging`,
|
||||
`PendingTable`, `PendingMode`, `StagedTablePath`,
|
||||
`dedupe_merge_batches_by_id`.
|
||||
- `crates/omnigraph/src/exec/mutation.rs` — D₂ check; per-op
|
||||
rewires (`execute_insert`, `execute_update`, `execute_delete*`);
|
||||
branch threading; coordinator-swap removal; helper
|
||||
`validate_edge_cardinality_with_pending`; helper
|
||||
`concat_match_batches_to_schema`; `apply_assignments` updated to
|
||||
copy unassigned blob columns from full-schema scans.
|
||||
- `crates/omnigraph/src/loader/mod.rs` — `load_jsonl_reader` split:
|
||||
staged path for Append/Merge, legacy inline-commit path for
|
||||
Overwrite. Helpers `collect_node_ids_with_pending` and
|
||||
`validate_edge_cardinality_with_pending_loader`.
|
||||
- `crates/omnigraph/src/table_store.rs` — `scan_with_pending`,
|
||||
`count_rows_with_pending` (DataFusion `MemTable`-backed union with
|
||||
Lance scan).
|
||||
- `Cargo.toml` (workspace) + `crates/omnigraph/Cargo.toml` — added
|
||||
`datafusion = "52"` direct dep (transitively pulled by Lance
|
||||
already; required for `MemTable`).
|
||||
- `docs/dev/writes.md` — removed "Known limitation" section; documented
|
||||
the new accumulator + D₂ + LoadMode::Overwrite residual.
|
||||
- `docs/dev/invariants.md` — mutation atomicity / read-your-writes status
|
||||
flipped to `upheld for inserts/updates`.
|
||||
- `docs/dev/architecture.md` — added "Mutation atomicity — in-memory
|
||||
accumulator" subsection; refreshed the engine + state
|
||||
diagrams to drop `RunRegistry` and add `MutationStaging`.
|
||||
- `docs/dev/execution.md` — rewrote the mutation flow sequence diagram
|
||||
for the staged-write path; updated the `LoadMode` table to call
|
||||
out per-mode commit semantics; rewrote `load` vs `ingest`.
|
||||
- `docs/user/query-language.md` — documented the D₂ parse-time rule.
|
||||
- `docs/user/errors.md` — added the D₂ `BadRequest` rejection path.
|
||||
- `docs/user/storage.md` — dropped the live `_graph_runs.lance` reference
|
||||
from the layout diagram and prose.
|
||||
- `docs/user/branches-commits.md` — moved `__run__<id>` to a legacy note;
|
||||
removed `publish_run` from the publish-trigger list.
|
||||
- `docs/user/audit.md` — current `_as` API list refreshed; legacy
|
||||
`RunRecord.actor_id` moved to a historical note.
|
||||
- `docs/user/constants.md` — marked the run registry / branch-prefix rows
|
||||
as legacy.
|
||||
- `docs/user/cli.md` — replaced the legacy `omnigraph run *` quickstart
|
||||
block with `omnigraph commit list/show`.
|
||||
- `docs/dev/testing.md` — extended the `writes.rs` row to cover the new
|
||||
staged-write contract tests; added the `staged_writes.rs` row.
|
||||
- `AGENTS.md` (CLAUDE.md symlink) — updated the atomic-per-query
|
||||
description and the L2 capability matrix row.
|
||||
|
||||
## Included Changes
|
||||
|
||||
- Rewire `mutate_as` and `load` via in-memory `MutationStaging` +
|
||||
`stage_*` / `commit_staged` per touched table at end-of-query.
|
||||
- (The storage substrate shipped in v0.4.0's PR #67 — `StagedWrite`,
|
||||
`stage_append`, `stage_merge_insert`, `commit_staged`,
|
||||
`scan_with_staged`, `count_rows_with_staged` — and is the substrate
|
||||
this release builds on.)
|
||||
Loading…
Add table
Add a link
Reference in a new issue