mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-07-12 03:12:11 +02:00
Implement RFC-022 unified graph write protocol (#343)
* Implement unified graph write protocol * Preserve recovery error wire compatibility
This commit is contained in:
parent
0c8d769501
commit
f758ff0d17
80 changed files with 13393 additions and 2050 deletions
|
|
@ -33,8 +33,10 @@ conflict kinds are on the [merge](merge.md) page.
|
|||
|
||||
## L2 — Recovery audit trail
|
||||
|
||||
Interrupted multi-table writes are recovered automatically the next time the graph is opened read-write. Recovery commits are recorded in the audit trail under the actor `omnigraph:recovery`, so you can find them with:
|
||||
|
||||
```bash
|
||||
omnigraph commit list --filter actor=omnigraph:recovery
|
||||
```
|
||||
Interrupted multi-table writes are recovered automatically the next time the
|
||||
graph is opened read-write. Each completed recovery is recorded internally in
|
||||
`_graph_commit_recoveries.lance`. A roll-forward keeps the interrupted
|
||||
writer's original commit id and actor; rollback and legacy recovery commits use
|
||||
the reserved actor `omnigraph:recovery`. Consequently, `commit list` is not a
|
||||
complete recovery log, and the CLI does not currently expose a query for the
|
||||
internal recovery-audit table.
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ Two primitives, two scopes:
|
|||
| **One `.gq` query** (any number of statements inside) | The query itself — handled by the publisher's atomic manifest commit | Yes — all statements land together or none of them do | The publisher never publishes; target unchanged |
|
||||
| **Many queries that must succeed together** | Branches: `branch_create` → run N queries on the branch → `branch_merge` | Yes — the merge is a single atomic publish | Drop the branch (`branch_delete`); main is unaffected |
|
||||
|
||||
Snapshot isolation is per-query — every read inside one query sees one consistent manifest version. Two concurrent queries on the same branch see independent snapshots; the publisher's CAS catches racing writes.
|
||||
Snapshot isolation is per-query — every read inside one query sees one consistent manifest version. Two concurrent queries on the same branch see independent snapshots. Mutation/load capture the branch head as coarse OCC authority, so a prepared plan is never silently reparented after another graph commit.
|
||||
|
||||
## Comparison with `BEGIN` / `COMMIT`
|
||||
|
||||
|
|
@ -136,7 +136,9 @@ This is the workflow agentic loops are designed around: **branches are the unit
|
|||
| Scenario | What happens | Caller action |
|
||||
|---|---|---|
|
||||
| Single query fails mid-flight | Publisher never publishes; target unchanged | Read the error, decide whether to retry |
|
||||
| Concurrent writers race the same `(table, branch)` | Publisher CAS rejects the loser with a version-mismatch conflict | Refresh handle, retry the query |
|
||||
| Branch authority changes before physical effects | Retryable inserts/loads fully reprepare; strict writes return `read_set_conflict` | For a surfaced strict conflict, refresh and retry deliberately |
|
||||
| Authority changes after a physical effect | The write returns `recovery_required` and leaves its durable sidecar | Resolve recovery by read-write reopen/server restart before retrying |
|
||||
| An overlapping recovery intent remains unresolved before effects | The write returns `recovery_required` with that intent's operation id and does not advance a table | Resolve recovery by read-write reopen/server restart before retrying |
|
||||
| Branch with N successful mutations, then merge fails (three-way conflict) | Each individual mutation already committed on the branch; merge surfaces `MergeConflicts` | Inspect, decide whether to keep working on the branch, abandon it (`branch_delete`), or resolve and re-merge |
|
||||
| Process crashes mid-branch-workflow | Each completed mutation on the branch is durable | Re-open the graph, continue where you left off |
|
||||
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ Top-level command families and subcommands. Graph-targeting commands accept a po
|
|||
| `schema plan \| apply \| show (alias: get)` | migrations. `apply` refuses a cluster-managed graph (one whose storage is inside a cluster) and points at `cluster apply` — those graphs evolve through the cluster ledger, not a direct apply |
|
||||
| `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` |
|
||||
| `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve <resource> --as <actor>` (`apply`/`approve` default the actor from `~/.omnigraph/config.yaml`'s `operator.actor` when `--as` is omitted); what apply converges is what an `omnigraph-server --cluster <dir>` deployment serves on its next restart (`--cluster` is the server's only boot source — cluster-only); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id |
|
||||
| `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) |
|
||||
| `optimize` | non-destructive Lance compaction + index reconciliation (blob-bearing tables use the normal path; tables with uncovered drift are skipped and `--json` reports `skipped`) |
|
||||
| `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review |
|
||||
| `cleanup --keep N --older-than 7d --confirm` | destructive version GC (`--confirm` to execute; also needs `--yes` against a non-local `s3://` target — see *Write diagnostics & destructive confirmation*) |
|
||||
| `embed` | offline JSONL embedding pipeline |
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ Every node type and every edge type is its own Lance dataset:
|
|||
- **Columnar Arrow storage**: each property is a column; nullable per Arrow schema.
|
||||
- **Fragments**: data is partitioned into fragments; new writes create new fragments.
|
||||
- **Manifest versioning**: every commit produces a new dataset version; old versions remain readable.
|
||||
- **Stable row IDs**: stable row IDs are enabled on every Lance dataset OmniGraph creates — node and edge data tables, `__manifest`, the commit-graph datasets, and any future system tables. This is an architectural invariant: the flag is one-way at dataset create, so a future change that introduces a Lance dataset must preserve it. Consequences: `_row_created_at_version` and `_row_last_updated_at_version` are available on every dataset (load-bearing for change-feed validators); indices survive `omnigraph optimize`. Pre-0.4.x graphs created before this code path settled may have datasets without the flag and cannot be retrofitted in place — the supported path is dump-and-reload. The rewrite path used by `schema_apply` preserves the flag.
|
||||
- **Stable row IDs**: stable row IDs are enabled on every Lance dataset OmniGraph creates — node and edge data tables, `__manifest`, `_graph_commit_recoveries.lance`, and any future system tables. This is an architectural invariant: the flag is one-way at dataset create, so a future change that introduces a Lance dataset must preserve it. Consequences: `_row_created_at_version` and `_row_last_updated_at_version` are available on every dataset (load-bearing for change-feed validators); indices survive `omnigraph optimize`. Pre-0.4.x graphs created before this code path settled may have datasets without the flag and cannot be retrofitted in place — the supported path is dump-and-reload. The rewrite path used by `schema_apply` preserves the flag.
|
||||
- **Append / delete / `merge_insert`**: native Lance write modes.
|
||||
- **Per-dataset branches** (Lance native): copy-on-write at the dataset level.
|
||||
- **Object-store agnostic**: file://, s3://, gs://, az://, http (read-only via Lance) — OmniGraph wires file:// and s3://.
|
||||
|
|
@ -30,7 +30,7 @@ OmniGraph is **not** a single Lance dataset; it is a *graph* of datasets coordin
|
|||
- **Snapshot reconstruction**: latest visible `table_version` per `(table_key, table_branch)` minus tombstones — rows where `object_type = table_tombstone`, whose own `table_version` (acting as the tombstone version) is `>= the entry's table_version`.
|
||||
- **Atomic publish**: multi-dataset commits publish so that a single write to `__manifest` flips all the new sub-table versions visible at once.
|
||||
- **Row-level CAS on the merge-insert join key**: `object_id` carries an unenforced-primary-key annotation so Lance's bloom-filter conflict resolver rejects two concurrent commits that land the same `object_id` row. Without this annotation, Lance's transparent rebase would admit silent duplicates from racing publishers.
|
||||
- **Optimistic concurrency control on publish**: a publish asserts the manifest's current latest non-tombstoned version for each touched table is exactly what the caller observed; mismatches surface as an `ExpectedVersionMismatch` manifest conflict naming the table and the expected/actual versions. Concurrent advances surface as a conflict rather than being silently rebased through.
|
||||
- **Optimistic concurrency control on publish**: legacy writers assert the manifest's current latest non-tombstoned version for each touched table; a mismatch surfaces as `ExpectedVersionMismatch`. RFC-022-enrolled mutation/load attempts use a stronger, branch-wide contract: preparation captures the Lance-native branch identity, the exact `graph_head` (including absence), the accepted schema identity/catalog, and one base table snapshot. Under root-shared schema → branch → sorted-table gates, the engine revalidates that complete authority before any physical effect, then the publisher rechecks the exact native branch identity/head plus the touched-table versions. An insert-only mutation or Append/Merge load whose authority changed before effects discards and fully reprepares the bounded attempt; Update/Delete/Overwrite returns `ReadSetChanged`. Once any Lance effect is durable, any later failure leaves the recovery sidecar authoritative and returns `RecoveryRequired` instead of silently rebasing or replaying the prepared plan.
|
||||
|
||||
### Internal schema versioning
|
||||
|
||||
|
|
@ -95,7 +95,7 @@ flowchart TB
|
|||
- **`__manifest/`** is a Lance dataset whose rows describe which sub-table version is published at which graph-branch. Reading a snapshot starts here.
|
||||
- **`nodes/`** and **`edges/`** are sibling directories holding one Lance dataset per declared type. Names are `fnv1a64-hex` of the type name to keep paths fixed-length and case-safe.
|
||||
- The graph commit DAG lives in **`__manifest`** as `graph_commit` / `graph_head` rows written in the publish CAS (RFC-013 Phase 7). The former `_graph_commits.lance` / `_graph_commit_actors.lance` lineage tables are retired — a graph this binary creates has neither.
|
||||
- **`_graph_commit_recoveries.lance`** — one row per crash-recovery action. Joined by `graph_commit_id` to the graph commit lineage (the `graph_commit` rows in `__manifest` since RFC-013 Phase 7); the linked commit carries `actor_id=omnigraph:recovery`. Operators correlate recoveries with the original mutations they rolled forward / back via this join.
|
||||
- **`_graph_commit_recoveries.lance`** — one internal row per completed crash-recovery action, including its exact per-table outcomes and the original actor. It joins by `graph_commit_id` to the graph commit lineage in `__manifest`. A v3 roll-forward keeps the interrupted writer's original actor; rollback and legacy recovery commits use `omnigraph:recovery`. The CLI does not currently expose this internal table.
|
||||
- **`__recovery/{ulid}.json`** — transient sidecar files written by a writer before it advances the underlying dataset, deleted once the matching manifest publish succeeds. A sidecar persisting after process exit means the writer crashed mid-commit; the next read-write open processes it. Steady-state directory is empty.
|
||||
- **`_refs/branches/{name}.json`** is graph-level branch metadata — pointers from a branch name to the manifest version it heads.
|
||||
- **Inside each Lance dataset** (orange): the standard Lance directory layout. `_versions/{n}.manifest` records every commit; `data/` holds the actual Arrow fragments; `_indices/{uuid}/` holds index segments with their own `fragment_bitmap` for partial coverage; `_refs/` holds Lance-native per-dataset branches and tags.
|
||||
|
|
|
|||
|
|
@ -21,6 +21,12 @@ properties in the assignment block (`insert WorksAt { person: $p, org: $o }`).
|
|||
|
||||
`<value>` is a literal, `$param`, or `now()`.
|
||||
|
||||
On a blob-bearing type, an update materializes and rewrites blob payloads only
|
||||
for the rows matched by its predicate, including blobs the update does not
|
||||
change. This keeps correctness independent of physical index state, but adds
|
||||
read/write I/O proportional to the matched blob bytes; use selective update
|
||||
predicates for large blobs.
|
||||
|
||||
## Atomicity
|
||||
|
||||
A change query publishes **one commit** at the end of the query. Multiple
|
||||
|
|
@ -29,6 +35,19 @@ failure leaves the graph untouched. See [transactions](../branching/transactions
|
|||
for the per-query atomicity contract and [branches](../branching/index.md) for
|
||||
multi-query workflows.
|
||||
|
||||
Concurrent changes use optimistic concurrency over the whole target branch.
|
||||
Insert/Merge/Append operations whose branch changed before physical effects are
|
||||
discarded and fully revalidated with a bounded internal retry. Strict
|
||||
Update/Delete/Overwrite operations instead return a structured conflict. This
|
||||
branch-wide token is deliberately conservative: a change to a different table
|
||||
can invalidate a prepared strict write because constraints may have read it.
|
||||
|
||||
If the synchronous barrier finds an unresolved overlapping recovery intent, or
|
||||
if a conflict is discovered after a Lance table effect is durable, the request
|
||||
returns `recovery_required` with an operation id. Do not immediately retry that
|
||||
request; reopen the graph read-write (or restart the server) so the durable
|
||||
recovery intent is resolved first.
|
||||
|
||||
## Inserts/updates and deletes cannot mix in one query
|
||||
|
||||
A single change query must be **either insert/update-only or delete-only**.
|
||||
|
|
|
|||
|
|
@ -31,13 +31,12 @@ List commits to see who made each change:
|
|||
omnigraph commit list graph.omni
|
||||
```
|
||||
|
||||
System-initiated writes use reserved actor ids — for example, automatic recovery
|
||||
of an interrupted write records `omnigraph:recovery`, so operator changes and
|
||||
machine repairs are distinguishable in the history:
|
||||
|
||||
```bash
|
||||
omnigraph commit list --filter actor=omnigraph:recovery graph.omni
|
||||
```
|
||||
System-initiated writes use reserved actor ids. Rollback and legacy recovery
|
||||
commits use `omnigraph:recovery`, while a v3 roll-forward preserves the
|
||||
interrupted writer's original commit id and actor. Exact recovery actions and
|
||||
per-table outcomes are stored in the internal
|
||||
`_graph_commit_recoveries.lance` audit table; the CLI does not currently expose
|
||||
that table, so `commit list` alone is not a complete recovery log.
|
||||
|
||||
## What is tracked
|
||||
|
||||
|
|
|
|||
|
|
@ -8,9 +8,11 @@
|
|||
- `Io(io::Error)`
|
||||
- `Manifest(ManifestError { kind: BadRequest|NotFound|Conflict|Internal, details: Option<ManifestConflictDetails>, … })`
|
||||
- `ManifestConflictDetails::ExpectedVersionMismatch { table_key, expected, actual }` — caller's `expected_table_versions` did not match the manifest's current latest non-tombstoned version (set by `OmniError::manifest_expected_version_mismatch`).
|
||||
- `ManifestConflictDetails::ReadSetChanged { member, expected, actual }` — an RFC-022 prepared write's branch/head/table authority changed before physical effects. HTTP returns **409** with `read_set_conflict`. A retry must start from preparation; strict writes leave that choice to the caller.
|
||||
- `ManifestConflictDetails::RowLevelCasContention` — Lance row-level CAS rejected the publish because a concurrent writer landed the same `object_id`. Retried internally by the publisher; only surfaces if the retry budget exhausts.
|
||||
- **D₂ parse-time rejection**: a single mutation query that mixes inserts/updates with deletes errors out *before any I/O* with kind `BadRequest`. Message: `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes`. See [query-language.md](../queries/index.md) for the rule.
|
||||
- `MergeConflicts(Vec<MergeConflict>)`
|
||||
- `RecoveryRequired { operation_id, reason }` — an overlapping durable recovery intent remains unresolved. Its physical effects may already have landed, or it may still be armed before the first effect. HTTP returns **503** with `recovery_required.operation_id`. Resolve the sidecar through a read-write reopen/server restart before retrying; this is intentionally not an ordinary OCC retry.
|
||||
|
||||
Compiler-side `CompilerError` covers parse / catalog / type / storage / plan / execution / arrow / lance / IO / manifest / unique-constraint, each with structured spans (`SourceSpan { start, end }`) for ariadne-style diagnostics. The legacy `NanoError` name remains as a deprecated compatibility alias.
|
||||
|
||||
|
|
|
|||
|
|
@ -9,13 +9,13 @@
|
|||
- **Also compacts the internal `__manifest` table** (RFC-013 step 2), which accumulates one fragment per commit — it now carries the graph lineage and actor rows inline (RFC-013 Phase 7: `graph_commit` / `graph_head` rows), so on the authenticated write path every commit's actor lands here too — and otherwise makes every write's metadata scan grow with history. (The `_graph_commits.lance` / `_graph_commit_actors.lance` tables are retired, so there is no separate lineage table to compact.) It takes a simpler path than data tables: `__manifest` is read at its latest version, so compaction just advances its version in place — **no manifest publish and no recovery sidecar**. (The sidecar-free property is not because it is one commit — `compact_files` can emit a `ReserveFragments` commit before the `Rewrite`, and the auto-cleanup strip below is a further commit — but because every one of those commits is content-preserving and the table is read at its latest version, so a crash at any point leaves it readable and content-identical and the next `optimize` re-plans.) It appears in the returned stats under `table_key` `"__manifest"`. It is **not yet covered by `cleanup`**, so its version chain still grows until the cleanup half lands (it requires a cleanup-resurrection safeguard first); run `optimize` on a cadence to keep per-write metadata scans flat.
|
||||
- **`optimize` is non-destructive by construction — it never garbage-collects versions, on any table (data or internal).** Compaction rewrites fragments and advances the version; old versions stay reachable until you run `cleanup`. This holds even for a graph created by an older binary that stored an on-by-default Lance `auto_cleanup` hook: `compact_files` / `optimize_indices` commit with the hook enabled and expose no skip override, so before compacting **any** table `optimize` strips its stale `lance.auto_cleanup.*` config first, so Lance's commit-time GC hook cannot fire and silently prune `__manifest`-pinned versions. (Graphs created by current binaries store no such config; the strip is the upgrade-path safety net.) The internal-table path additionally tolerates a concurrent live writer: it runs a **bounded** rebase-and-retry, so transient contention does not fail the operator's `optimize` or the live write — but sustained contention past the retry budget surfaces a loud conflict error rather than looping forever (bounded and observable, not a silent give-up). The data-table path holds the per-table write queue while it compacts, so it does not contend with mutations on that table in the first place.
|
||||
- **Reindex (index coverage maintenance).** A scalar/FTS/vector index only covers the fragments it was built over. Rows appended after the index was built (e.g. by `load --mode merge`, whose commit does not rebuild an already-existing index) are scanned unindexed, and compaction itself rewrites fragments out of an index's coverage. `optimize` runs Lance's incremental `optimize_indices` after compaction to fold those fragments back in (a delta merge, not a full retrain), restoring full coverage so equality/range/traversal predicates stay index-accelerated. This is why a table with **no compaction work but stale index coverage still commits** a new version under `optimize`. Run `optimize` on a cadence at least as frequent as your freshness window so recently-loaded rows do not linger in the unindexed flat-scan tail.
|
||||
- **Create declared-but-missing indexes (the index reconciler).** `@index`/`@key` declares intent; `schema apply` records it but builds nothing, and `load`/`mutate` defer a column that cannot be built yet (a `Vector` column with no trainable vectors). `optimize` materializes any such declared-but-unbuilt index over the compacted layout — so it is the convergence path for an `@index` added after data exists, or a vector index whose embeddings arrived via a later `embed`. A column still not buildable (no vectors yet) is reported on the table's stat as `pending_indexes` (visible in `--json`), not treated as a failure; the next `optimize` retries. So `optimize` is the single operator-facing index reconciler: it compacts, restores coverage, **and** builds declared-but-missing indexes.
|
||||
- **Create declared-but-missing indexes (the index reconciler).** `@index`/`@key` declares intent; `schema apply`, `load`, and `mutate` build no physical indexes inline. They record or publish only their exact logical/data effects and leave all index materialization to `ensure_indices`/`optimize`. `optimize` materializes every buildable declared-but-missing index over the compacted layout — so it is the convergence path for an `@index` added after data exists, or a vector index whose embeddings arrived via a later `embed`. A column still not buildable (no vectors yet) is reported on the table's stat as `pending_indexes` (visible in `--json`), not treated as a failure; the next `optimize` retries. So `optimize` is the single operator-facing index reconciler: it compacts, restores coverage, **and** builds declared-but-missing indexes.
|
||||
- Each table's compact→reindex→publish serializes with concurrent mutations on the same table. A crash mid-operation is recovered automatically on the next open (both compaction and reindex are content-preserving, so roll-forward is always safe).
|
||||
- **Requires a recovered graph.** `optimize` refuses (errors) when a pending crash-recovery operation is present — operating on an unrecovered graph could publish a partial write that recovery would roll back. Reopen the graph to run recovery, then re-run `optimize`.
|
||||
- **Uncovered drift is skipped, not interpreted.** If a table's underlying version is ahead of the version recorded in `__manifest` and no crash-recovery record covers that movement, `optimize` reports `skipped: DriftNeedsRepair` with the manifest/head versions and leaves the table untouched. Run `omnigraph repair` to classify and explicitly publish that drift.
|
||||
- Bounded by `OMNIGRAPH_MAINTENANCE_CONCURRENCY` (default 8).
|
||||
- Returns per-table stats: `table_key, fragments_removed, fragments_added, committed, skipped, manifest_version, lance_head_version, pending_indexes` (the last lists any declared `@index` column the reconciler could not build this run, with the reason — e.g. a vector column with no trainable vectors yet).
|
||||
- **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: BlobColumnsUnsupportedByLance` (and logged) instead of compacted, and the rest of the sweep proceeds normally. **Reads and writes are unaffected** — only compaction is. Consequence: fragment count and deleted-row space on blob tables are not reclaimed; query results are never affected. A skipped blob table is also **not reindexed** in the same sweep (the skip happens before the reindex step), so its index coverage on appended rows is not refreshed by `optimize` today.
|
||||
- **Blob tables use the normal compaction and reindex path.** Lance 8.0.0+ supports blob-v2 compaction, so OmniGraph no longer has a blob-specific skip or capability gate. Fragment reclamation and index-coverage repair therefore apply to blob-bearing tables like every other table.
|
||||
|
||||
## `repair` — explicit
|
||||
|
||||
|
|
@ -39,7 +39,12 @@
|
|||
- CLI guards with `--confirm`; without it, prints a preview line.
|
||||
- **Non-local consent.** Against a non-local target (an `s3://` store/cluster), `cleanup` additionally requires `--yes` on top of `--confirm`: a TTY is prompted, and a non-interactive run (no TTY, or `--json`) refuses rather than destroying. A local (`file://`) target needs only `--confirm`. The same `--yes` gate applies to overwrite `load` and `branch delete`; every maintenance run echoes its resolved target to stderr (suppress with `--quiet`).
|
||||
- **Recovery floor:** `--keep < 3` may garbage-collect versions that crash recovery needs as a rollback target. Default `--keep 10` is safe.
|
||||
- **Orphaned-branch reconciliation:** before the version GC, cleanup reclaims any per-table or commit-graph branch absent from the manifest branch list. These orphans arise when a `branch_delete` flips the manifest authority but a downstream best-effort reclaim does not complete (see [branches-commits.md](../branching/index.md)). The reconciler is idempotent (it no-ops once nothing is orphaned), runs regardless of the `keep_versions` / `older_than` values (those gate version GC only), and never reclaims `main` or system-branch forks. Reclaimed forks are logged.
|
||||
- **Requires clean recovery state.** If any durable recovery intent is pending,
|
||||
cleanup refuses before orphan reconciliation or version GC. Reopen the graph
|
||||
read-write (or restart the server) to resolve recovery, then rerun cleanup;
|
||||
deleting transaction/version history while an intent is pending would make
|
||||
exact effect ownership unverifiable.
|
||||
- **Orphaned-branch reconciliation:** before the version GC, cleanup reclaims any per-table Lance branch absent from the manifest branch list. These orphans arise when a `branch_delete` flips the manifest authority but a downstream best-effort reclaim does not complete (see [branches-commits.md](../branching/index.md)). The reconciler is idempotent (it no-ops once nothing is orphaned), runs regardless of the `keep_versions` / `older_than` values (those gate version GC only), and never reclaims `main` or system-branch forks. Reclaimed forks are logged. Graph lineage has no separate branch dataset: it lives in `__manifest`.
|
||||
|
||||
## Tombstones
|
||||
|
||||
|
|
|
|||
|
|
@ -162,25 +162,51 @@ Only `/export` streams (`application/x-ndjson`, MPSC channel + `Body::from_strea
|
|||
|
||||
## Error model
|
||||
|
||||
Uniform `ErrorOutput { error, code?, merge_conflicts[], manifest_conflict? }` with `code ∈ unauthorized | forbidden | bad_request | not_found | conflict | too_many_requests | internal`. Merge conflicts attach structured `MergeConflictOutput { table_key, row_id?, kind, message }`.
|
||||
Uniform
|
||||
`ErrorOutput { error, code?, merge_conflicts[], manifest_conflict?, read_set_conflict?, recovery_required? }`
|
||||
with
|
||||
`code ∈ unauthorized | forbidden | bad_request | not_found | method_not_allowed | conflict | too_many_requests | internal`.
|
||||
Merge conflicts attach structured
|
||||
`MergeConflictOutput { table_key, row_id?, kind, message }`.
|
||||
|
||||
`manifest_conflict` is set on **concurrent-write rejections** (HTTP 409): the
|
||||
caller's pre-write view of one table's manifest version was stale.
|
||||
`ManifestConflictOutput { table_key, expected, actual }` tells the client
|
||||
which table to refresh and retry. This is the conflict shape produced by
|
||||
concurrent `/mutate` (or its `/change` alias), `/load` (or its deprecated
|
||||
`/ingest` alias) calls landing the same `(table, branch)` race.
|
||||
`manifest_conflict` is set on legacy per-table manifest-version rejections
|
||||
(HTTP 409). `ManifestConflictOutput { table_key, expected, actual }` tells the
|
||||
client which table was stale. Mutation and load use the unified coarse-OCC
|
||||
adapter described next; other writers retain this older conflict shape until
|
||||
they are enrolled.
|
||||
|
||||
HTTP status codes used: 200, 400, 401, 403, 404, 409, 429, 500.
|
||||
`read_set_conflict` is set when a prepared write is rejected before any table
|
||||
effect because its branch authority changed. The HTTP status is 409 and
|
||||
`ReadSetConflictOutput { member, expected, actual }` identifies the stale
|
||||
authority member. The engine already performs a bounded full-attempt retry for
|
||||
mutation inserts and load `append`/`merge`. Strict mutation updates/deletes and
|
||||
load `overwrite` return the 409 to the caller instead of being replayed.
|
||||
|
||||
`recovery_required` is set when an overlapping durable recovery intent remains
|
||||
unresolved; its table effects may or may not have started. The HTTP status is 503 and
|
||||
`RecoveryRequiredOutput { operation_id }` names the durable recovery intent.
|
||||
The optional `code` field is omitted for this response: adding a new value to
|
||||
the closed error-code enum would break older clients, while the optional
|
||||
structured field is additive and rolling-safe.
|
||||
Do not blindly resubmit the write: let a read-write open or the recovery sweep
|
||||
resolve that operation first, then retry from a fresh snapshot.
|
||||
|
||||
HTTP status codes used: 200, 400, 401, 403, 404, 405, 409, 429, 500, 503.
|
||||
|
||||
## Per-actor admission control
|
||||
|
||||
Disjoint
|
||||
`(table, branch)` writes from different actors now run concurrently,
|
||||
guarded only by the engine's per-(table, branch) write queue. To keep
|
||||
one heavy actor from exhausting shared capacity (Lance I/O, manifest
|
||||
churn, network), the server gates mutating handlers through per-process
|
||||
admission limits configured from environment variables:
|
||||
RFC-022-enrolled mutation/load preparation runs outside the effect gates, so
|
||||
parsing, validation, and reclaimable fragment staging can overlap across branches.
|
||||
Readers acquire none of these gates. Before the first durable effect, however, an
|
||||
attempt acquires the exclusive root schema gate, then its branch-effect gate and
|
||||
sorted table queues, and holds all of them through manifest publication. The root
|
||||
schema gate means enrolled effect windows on one graph currently serialize
|
||||
in-process even across different branches; the branch gate preserves one atomic
|
||||
graph-head validation authority, while table queues protect each concrete Lance
|
||||
effect and legacy writer. These are process-local ordering gates, not a
|
||||
cross-process lock. To keep one heavy actor from exhausting shared capacity
|
||||
(Lance I/O, manifest churn, network), the server gates mutating handlers through
|
||||
per-process admission limits configured from environment variables:
|
||||
|
||||
| Env var | Default | Purpose |
|
||||
|---|---|---|
|
||||
|
|
|
|||
|
|
@ -4,14 +4,13 @@
|
|||
|---|---|---|
|
||||
| `MANIFEST_DIR` | `__manifest` | manifest layout |
|
||||
| Commit graph dirs (retired) | `_graph_commits.lance` / `_graph_commit_actors.lance` | retired in Phase B; lineage lives in `__manifest` (`graph_commit` / `graph_head` rows) since RFC-013 Phase 7. A graph this binary creates has neither. |
|
||||
| Recovery audit dir | `_graph_commit_recoveries.lance` | one row per crash-recovery action (`omnigraph commit list --filter actor=omnigraph:recovery`) |
|
||||
| Recovery audit dir | `_graph_commit_recoveries.lance` | internal exact record of completed crash-recovery actions; no public CLI query yet |
|
||||
| Run branch prefix (legacy, removed) | `__run__` | pre-v0.4.0 Run state machine; no longer a reserved name. A graph still carrying `__run__*` branches is sub-v4 and refused on open (rebuild via export/import). |
|
||||
| Schema apply lock | `__schema_apply_lock__` | schema apply |
|
||||
| Manifest publisher retry budget | `PUBLISHER_RETRY_BUDGET = 5` | manifest publish |
|
||||
| Internal manifest schema version | `INTERNAL_MANIFEST_SCHEMA_VERSION = 4` | manifest migrations (v4 = graph lineage in `__manifest`, RFC-013 Phase 7) |
|
||||
| Merge stage batch | `MERGE_STAGE_BATCH_ROWS = 8192` | merge execution |
|
||||
| Maintenance concurrency | `OMNIGRAPH_MAINTENANCE_CONCURRENCY=8` | optimize/cleanup |
|
||||
| Lance blob compaction support | `LANCE_SUPPORTS_BLOB_COMPACTION = false` | optimize |
|
||||
| Graph index cache size | `8` (LRU) | runtime cache |
|
||||
| Expand indexed-path frontier ceiling | `OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER=1024` | traversal |
|
||||
| Expand indexed-path hop ceiling | `OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS=6` | traversal |
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ list/`Blob` columns → none.
|
|||
|
||||
## L2 — OmniGraph orchestration
|
||||
|
||||
- **`@index`/`@key` declares intent; the physical index is derived state.** A migration records the declaration in the catalog/IR and never fails on it — `schema apply` builds **no** indexes (adding an `@index` to an existing column is a pure metadata change that touches no table data). `load`/`mutate` build declared indexes inline as part of the write, but a column that can't be built yet (a `Vector` column with no trainable vectors — IVF k-means needs ≥1 vector, e.g. rows loaded before `embed` runs) is left **pending**, not fatal. Reads stay correct meanwhile: a missing/partial index degrades to a scan (vector search to brute-force). A later `ensure_indices`/`optimize` materializes the pending index once it is buildable. This mirrors how LanceDB builds indexes asynchronously and serves unindexed rows by brute-force.
|
||||
- **`@index`/`@key` declares intent; the physical index is derived state.** A migration records the declaration in the catalog/IR and never fails on it — `schema apply` builds **no** indexes. Mutation/load likewise publish only their exact data effects; they do not widen the recovery plan with index commits. Reads stay correct while an index is missing or partially covered by falling back to scans (vector search to brute-force). A later `ensure_indices`/`optimize` materializes every buildable declaration; an untrainable Vector column remains pending rather than failing the logical write.
|
||||
- `ensure_indices()` / `ensure_indices_on(branch)` — idempotent build of BTREE + inverted + vector indexes for the current head; safe to re-run; returns the columns it had to defer as pending. `optimize` runs it after compaction, so the maintenance cron is the convergence path for deferred indexes.
|
||||
- Indexes are built on the *branch head* (not on a snapshot), so reads always see the current index state.
|
||||
- **Lazy branch forking for indexes**: a branch that hasn't mutated a sub-table doesn't need its own index — the main lineage's index is reused until the first write triggers a copy-on-write fork.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue