Four guards/refactors that convert previously convention-enforced MCP
invariants into ones a future edit can't silently break:
- H1 (loopback Host set): standalone.rs surface-guard asserts our loopback
allow-list is a superset of rmcp's own default set, so an rmcp bump that
adds a loopback form turns red instead of 403'ing that client. Pins the
::1 regression by construction rather than by a hand-kept literal list.
- H2 (one stored-query gate): extract a single invoke_query_request() in
handlers.rs used by GET /queries, POST /queries/{name}, and (imported) the
MCP tools/list + tools/call stored paths. REST and MCP can no longer drift
on which Cedar action governs the catalog. Deletes mcp.rs's local copy.
- H3 (expose chokepoint): tests/mcp.rs source-walk guard bans .lookup( and
registry.iter( in mcp.rs, so the agent surface can only reach stored
queries through exposed()/exposed_by_name() — an @mcp(expose:false) query
cannot leak back into tools/list via the expose-ignoring registry methods.
- H4 (list-gate relaxation lower-bound): a permit-all actor must see every
built-in tool, pinning the other end of the list-gate fix (no callable
tool may be hidden from tools/list).
cargo test -p omnigraph-mcp -p omnigraph-server green (the lone schema_routes
flake was disk-pressure during the clean build; passes in isolation).
Bring the MCP feature branch up to date with main (14 commits). One
conflict — compiler/parser.rs: main's `NanoError` → `CompilerError` rename
vs this branch's `@mcp` / per-param `@description` parser additions; resolved
by keeping the new parsing under the renamed error type. The CLI `queries list`
change (#280, surfacing `@description`/`@instruction`) auto-merged with this
branch's `mcp_expose`/`tool_name` columns.
* test(engine): regression tests for #283 camelCase property filters
Red against current code. A query (or chained mutation) that filters on a
camelCase schema field lints and plans cleanly but fails at run time with
"No field named reponame" because the identifier's case is destroyed at the
engine->Lance boundary.
Coverage added:
- query.rs unit: ir_filter_to_expr on a camelCase property must emit an
Expr::Column named `repoName`, not `reponame` (red); plus a green coercion
guard that a camelCase int column still gets a coerced literal.
- mutation.rs unit: predicate_to_sql must emit the column UNQUOTED and
case-preserved (green guard documenting the committed-scan contract).
- literal_filters.rs e2e: a camelCase @index field with an inline-binding
pushdown filter returns the seeded row (red — read pushdown).
- writes.rs e2e: an update+delete on a camelCase predicate, and a chained
update that re-reads the pending side of scan_with_pending by the same
camelCase predicate (red — pending MemTable scan).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7
* fix(engine): preserve identifier case in filter pushdown (#283)
Two engine->Lance boundaries lowercased camelCase column identifiers,
breaking any filter on a camelCase schema field even though the IR,
compiler, projection, and in-memory filtering all preserve case.
Read pushdown (exec/query.rs, ir_expr_to_expr): build the column reference
with datafusion::prelude::ident() instead of col(). col() routes through SQL
identifier normalization and lowercases an unquoted identifier
(`repoName` -> `reponame`); ident() builds an unqualified, case-preserved
Column. Property refs here are always bare column names, so there is no
qualified-name handling to lose. No-op for the lowercase columns that work
today.
Pending mutation scan (table_store.rs, scan_pending_batches): the
committed-scan consumer (Lance Scanner::filter(&str)) preserves an unquoted
identifier's case but treats a double-quoted "col" as a string literal, so
predicate_to_sql must keep the column unquoted. The pending side splices that
same unquoted predicate into a DataFusion `SELECT ... WHERE`, which would
lowercase it. Make that path case-preserving by disabling
sql_parser.enable_ident_normalization on its SessionContext rather than
quoting (quoting would match zero committed rows). predicate_to_sql gains
only a clarifying comment; its emitted string is unchanged.
Full engine suite green (579 tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7
* docs(dev): case study for #283 camelCase filter bug
Record the root cause, the two-boundary fix (read pushdown col→ident; pending
mutation scan ident-normalization off), and why the obvious symmetric
"quote the column" fix is wrong (Lance reads a double-quoted column as a string
literal and silently matches zero committed rows). Linked from a new
"Case Studies" section in the dev index so the link check passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: e2e coverage for @description/@instruction surfaces
Add end-to-end tests pinning the two annotation surfaces as they exist
today, at their real boundaries:
- engine (lifecycle.rs): schema-level @description (node/edge/property)
and @instruction (node/edge) persist verbatim into the on-disk
_schema.ir.json through Omnigraph::init; property-level @instruction
aborts init and writes no schema IR.
- server (stored_queries.rs): query-level @description/@instruction on a
stored query surface as typed QueryCatalogEntry fields over
GET /queries, and a query declaring neither omits both fields.
No behavior change — these document the current contract.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cli): surface stored-query @description/@instruction in `queries list`
A stored query's @description/@instruction are its catalog metadata —
what it does and how to invoke it. The HTTP GET /queries catalog already
carries them, but `omnigraph queries list` dropped both fields in human
and --json output even though they were available on the registry entry.
Carry description/instruction on QueriesListItem (Option, skipped when
None) and copy them from the query decl. Human output prints an indented
`description:` / `instruction:` line per query when present; --json
includes the fields when present and omits them otherwise — matching the
HTTP catalog shape documented in docs/user/operations/server.md.
Tests (cli_queries.rs): a query with both annotations surfaces them in
human + --json; a query with neither prints no annotation lines and omits
both JSON fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(cli): document `queries list` output incl. description/instruction
Per AGENTS.md maintenance Rule 1, document the user-visible `queries list`
output alongside the field addition. The `queries` command family had no
row in the CLI reference top-level table; add one covering `list` (human
+ --json shapes, with description/instruction shown only when declared,
matching the HTTP GET /queries catalog) and `validate`.
Addresses the Greptile P2 review finding on PR #280.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): indent multiline stored-query annotations in `queries list`
A `@description`/`@instruction` value can be multiline (GQ string literals
admit newlines), which made the human `queries list` output break back to
the left margin on continuation lines. Indent continuation lines to align
under the first via a `print_query_annotation` helper. Addresses review
feedback from @martin-g on PR #280.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Ragnor Comerford <ragnor.comerford@gmail.com>
* fix(cluster): stop cluster-apply crash-loops from the recovery-sidecar trap
A `cluster apply` carrying a schema change against a graph that has
non-main branches, or an unsupported "needs backfill" migration, armed a
recovery sidecar *before* calling the engine, then left it behind when the
engine rejected the apply pre-movement. The server refuses to boot while
any sidecar is pending, and re-running apply re-armed a fresh sidecar — an
unescapable crash loop. None of the engine rejections are bugs; the trap
is in the apply/serve choreography.
Three coordinated changes:
1. Preview before arming the sidecar. `cluster apply` now runs
`preview_schema_apply_with_options` before `write_recovery_sidecar`, so
parser/planner rejections (non-main branches, unsupported plan) fail
loudly without leaving recovery work behind. The post-preview engine
error path now deletes the sidecar when the live schema still matches
the recorded digest (nothing moved), and keeps it only on real
mid-movement failure — both branches covered by new engine-failpoint
tests (cluster failpoints now enable omnigraph/failpoints).
2. Per-graph quarantine at serve time instead of whole-cluster refusal.
A graph-attributed pending sidecar, an unopenable graph root, a query
parse failure, or an unresolvable embedding provider now quarantines
just that graph (logged loudly at every boot layer) while healthy
graphs serve; `/graphs` lists only ready graphs and quarantined routes
404. Cluster-global problems (missing/unreadable state, malformed or
unattributable sidecars, shared-catalog or cluster-policy errors, zero
healthy graphs) stay fail-fast. `--require-all-graphs` /
OMNIGRAPH_REQUIRE_ALL_GRAPHS=1 restores all-or-nothing boot.
3. Backfill embedding-provider profile metadata on apply. Mirrors the
existing policy-binding backfill: a pre-5A ledger missing
`embedding_profile` is now detected as a metadata-only change and
backfilled by a no-op apply, instead of bricking serve with
`embedding_provider_profile_missing` forever.
Tests: trap (no sidecar after a rejected apply), both digest-cleanup
branches, per-graph quarantine (cluster + server), embedding backfill.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: resilient cluster boot + recovery-sidecar trap fix
Amend RFC-005 D4 readiness posture (cluster-global fail-fast vs graph-local
quarantine; deviation #5 for --require-all-graphs), add the v0.7.0 release
note, and update the user cluster/server/deployment docs and the
OMNIGRAPH_REQUIRE_ALL_GRAPHS env var.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cluster): surface sidecar-cleanup failures; document severity promotion
Address Greptile review on PR #284:
- The pre-movement sidecar cleanup fast-path discarded `delete_object`'s
result, so a transient delete failure left the graph quarantined with no
signal. Add `try_delete_object` (Result-returning) and emit a
`recovery_sidecar_cleanup_failed` warning diagnostic on failure; the
fire-and-forget `delete_object` now delegates to it.
- Document why the serve-time loop promotes every `list_recovery_sidecars`
diagnostic to a cluster-fatal error (the listing only emits genuine
read/parse/version failures, as warnings, whose blast radius serving
cannot prove) and note the promote-by-code path if that ever changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(engine): stop branch-merge fast-forward OOM on embedding tables
A branch→main fast-forward merge of a forked, embedding-bearing table
re-derived the whole branch row-by-row: it lumped new + changed rows into
one Lance `merge_insert`, i.e. a full-outer hash join over the entire
delta that exhausts the DataFusion memory pool (8k rows × 3072-dim →
`Resources exhausted: 188MB HashJoinInput, 100MB pool`), so the merge
hung/failed instead of completing.
Fix the data path on existing, substrate-supported primitives:
- Adopt-with-delta split: new rows → `stage_append` (a streaming
`Operation::Append`, no hash join), only genuinely-changed rows →
a bounded `stage_merge_insert`, deletes inline. New `AdoptDelta` /
`compute_adopt_delta` / `publish_adopted_delta` replace the combined
`compute_source_delta` path; the three-way merge path is untouched.
- Stream the append via `stage_append_stream` →
`execute_uncommitted_stream` (the substrate-blessed bulk-append path),
removing the `Vec`+`concat` full-delta materialization. Blob-aware via
`scan_stream_for_rewrite`. Exposed on the sealed `TableStorage` trait.
- Lazy row-signature: stop stringifying every row's embedding eagerly;
compute the signature only for the `(Some,Some)` changed-candidate arm.
- Index coverage is reconciler-owned: the adopt path no longer rebuilds
vector/FTS indexes inline; `optimize`/`ensure_indices` folds the new
rows in (reads stay correct via brute-force tail). Post-merge
index-coverage contract documented in docs/user/branching/merge.md.
- Recovery pin: new `CandidateTableState::AdoptWithDelta` is classified
and pinned so the append's HEAD advance is sidecar-covered
(invariant 5); the `BranchMerge` sidecar's loose classification covers
the two-commit shape.
The regression gate is structural, not a brittle size threshold: task-local
write probes assert an append-only fast-forward merge does 0
`stage_merge_insert` (the OOM hash join), appends via `stage_append`, and
streams (0 whole-delta materialization). Plus functional correctness,
blob round-trip, index-defer, and a Phase-B failpoint recovery test.
Residual: the classify-time staging round-trip is still O(N) in memory
(architecturally required for the all-or-nothing multi-table publish);
bounding it fully is the fragment-adopt follow-up.
* test(engine): partial branch-merge Phase B must roll back (RED regression)
A branch-merge per-table publish is a multi-commit sequence — adopt:
append → upsert → delete; three-way: merge_insert → delete → index — each
step advancing Lance HEAD before the single manifest publish. Add four
failpoint sites at those windows and four regression tests (mixed delta:
a fresh id, a modified base id, a removed base id) asserting that a crash
mid-sequence rolls the whole merge BACK on the next open and a re-run
re-applies the full delta.
RED against current code: the loose `BranchMerge` classification rolls any
`lance_head > manifest_pinned` forward, so the partial is published and the
merge recorded — the rolled-back-to-base assertion fails with the partial
state visible (e.g. bob appended, dave not deleted). The fix lands next.
The failpoint sites are no-ops unless the `failpoints` feature activates them.
* fix(engine): roll back partial branch-merge Phase B (recovery WAL confirmation)
A branch-merge publishes each table with several Lance commits (adopt:
append → upsert → delete; three-way: merge_insert → delete → index), then
one manifest publish makes them atomic. Recovery classified `BranchMerge`
loosely: any `lance_head > manifest_pinned` with a matching CAS pin rolled
*forward* to the observed HEAD. So a crash mid-sequence published a partial
delta (e.g. the append without its sibling upsert/delete) and recorded the
merge as complete — silent data loss; a re-merge sees "already up to date"
and never repairs it.
Fix: make the recovery sidecar a two-phase WAL for `BranchMerge`. After the
whole per-table publish loop completes, stamp each pin's `confirmed_version`
with its exact achieved Lance version (a second sidecar write), then publish
the manifest. Recovery now:
- rolls FORWARD only to a pin's `confirmed_version` (set ⇒ Phase B finished);
- rolls BACK (`TableClassification::IncompletePhaseB`) when the HEAD moved but
no confirmation was recorded ⇒ a partial publish ⇒ all-or-nothing restore to
the manifest pin, so a re-run re-applies the full delta.
Scope: `BranchMerge` only. Other loose writers (`SchemaApply`,
`EnsureIndices`, `Optimize`) keep the loose roll-forward — their drift is
derived state (index coverage, compaction) a partial roll-forward never
corrupts, so confirmation would be cost without benefit.
This is the write-ahead intent record + idempotent roll-forward that the
fast-forward-main commit model requires to be crash-atomic across N tables;
version-recorded (not phase-count-derived), so it survives later changes to
the per-table commit sequence.
Regression tests (failpoints): four partial-window crashes — adopt
after-append / after-upsert, three-way after-merge / after-delete — each with
a mixed delta (new id, modified id, removed id) now roll the whole merge back;
the existing complete-Phase-B tests still roll forward.
* fix(engine): scope merge index docs to fast-forward; record append probe after write
Two PR-review fixes:
- docs(merge): the "a merge does not build indexes inline" note only holds for
the fast-forward / adopt path (deferred to the reconciler). The three-way
`Merged` path still rebuilds indexes inline in its publish, so a
Merged-outcome merge of an embedding table pays the build up front. Scope the
doc so a Merged-outcome user isn't surprised or led to skip a post-merge
optimize.
- `stage_append` recorded its instrumentation probe before the fallible
`execute_uncommitted`, so a failed staging write left the call/row counters
inflated — and diverged from `stage_append_stream`, which records after the
transaction is built. Record after the write succeeds.
* fix(engine): record stage_merge_insert / vector-index probes after write too
The prior commit moved `stage_append`'s instrumentation probe to after the
write, but left the two sibling write primitives with the identical ordering
bug: `stage_merge_insert` recorded before `execute_uncommitted`, and
`create_vector_index` before the index build. A failed write on either would
inflate the probe counter. Move both to record only after the write succeeds,
so all write-primitive probes share one rule (record-after-success) — closing
the class rather than the single instance the review flagged.
* docs(engine): mark the fragment-adopt excision boundary in the merge code
Comment the transitional row-level merge code so a future fragment-adopt
implementation (Lance branch-merge/rebase #7263 + UUID branch paths #7185)
knows exactly what it deletes and what it keeps:
- `AdoptDelta` / `compute_adopt_delta` / `publish_adopted_delta` — the row-level
re-derivation; removed wholesale when a fast-forward merge becomes a fragment
graft (adopt the source table version's fragments + indexes by reference).
- `stage_append_stream` — its only caller is that merge append; dead with it
unless re-adopted as a general bulk-append path.
- `confirm_sidecar_phase_b` — the boundary marker: this SURVIVES. The recovery
sidecar is the cross-table WAL a fast-forward-main commit still needs; only the
within-table multi-commit reason for `IncompletePhaseB` narrows once each table
is a single graft commit. Keep the sidecar; only simplify the classifier.
Comments only; no behavior change.
* test(engine): pre-upgrade v1 branch-merge sidecar must roll forward (RED)
Phase-B confirmation made the recovery classifier strict for every BranchMerge
sidecar — including ones written by a binary that predates confirmation. A
pre-upgrade crash in the Phase-B→C gap can leave such a sidecar over a COMPLETED
merge; the new classifier reads its absent confirmed_version as a partial and
rolls it back, silently discarding the finished merge (greptile P1 / Cursor High).
This regression test synthesizes that sidecar realistically: crash after Phase B
(real sidecar + advanced Lance HEAD), downgrade the on-disk JSON to the
pre-confirmation v1 shape (schema_version=1, strip confirmed_version), reopen.
RED: the merge rolls back, `bob` is discarded (left ["alice"], want
["alice","bob"]). The versioning fix lands next.
* fix(engine): version the recovery sidecar; read pre-confirmation merges as loose
Phase-B confirmation changed how a BranchMerge sidecar's absent confirmed_version
is interpreted (roll forward → roll back) without versioning the artifact, so the
new classifier silently discarded completed pre-upgrade merges (greptile P1 /
Cursor High). A capability flag would not fix the symmetric direction — keeping
schema_version=1, an OLD binary reading a NEW sidecar sails through its
already-shipped strict gate, ignores the unknown flag, and applies loose
semantics to a new partial → the same data loss on downgrade. Use the versioning
system instead.
- Bump SIDECAR_SCHEMA_VERSION 1 → 2; add a fixed CONFIRMATION_SCHEMA_VERSION = 2
(the generation at which confirmation shipped — pinned, so a later v3 keeps v2
confirmation-aware).
- Make the read gate version-aware (`parse_sidecar`): refuse only versions NEWER
than this binary; accept and interpret older ones with their original
semantics — no operator toil draining pre-upgrade sidecars. Rename
`SidecarSchemaError.supported_version` → `max_supported_version` and reword.
- Dispatch classification by version: the strict BranchMerge confirmation path is
gated on `schema_version >= CONFIRMATION_SCHEMA_VERSION`; a v1 BranchMerge
sidecar falls through to the existing loose roll-forward. Thread
`sidecar.schema_version` from `process_sidecar`.
This is bidirectionally safe: a new binary interprets v1 (loose) and v2 (strict)
and refuses the future; an old binary's `!= 1` gate already refuses v2, so it
never misreads a new sidecar. The flag was an additive-field pattern misapplied
to a semantics change; versioning is the correct mechanism.
Honest residual (any approach): an old *partial* sidecar still rolls forward —
v1 carries no confirmation, so partialness is undetectable in it. The fix stops
us from interpreting old sidecars under new rules; it can't retrofit information
they never had.
* fix(engine): harden recovery — mode resolver, loud divergence check, publish classified version
Three correct-by-design fixes from the holistic review of the recovery path, all
in recovery.rs (each closes a class, not an instance):
A. Resolve the classification mode from `(kind, schema_version)` once, instead of
a kind×version match accreting fall-through guards in `classify_table`. New
`ClassificationMode { Strict, Loose, Confirmed }` + an exhaustive
`SidecarKind::classification_mode` — adding a writer kind or version floor is
now one arm in the resolver (the compiler forces it), not a guard threaded
through the classifier. No behavior change; existing classify/decide tests are
the guard.
B. `confirm_sidecar_phase_b` now errors loudly when a pinned table has no achieved
version in the publish `updates`, instead of silently skipping it (which left
the pin unconfirmed → `IncompletePhaseB` → a silent rollback of a COMPLETE
merge). Guards the implicit `pins ⊆ updates` invariant against a future
divergence between the two filters (invariants 9/13). + a unit test.
C. Recovery roll-forward publishes the version classification OBSERVED
(`state.lance_head`), not a fresh HEAD re-read at publish time. For a Confirmed
pin classify already validated `lance_head == confirmed_version`, so this
publishes the recorded WAL intent by construction and closes the
classify→publish re-derivation/TOCTOU for every writer (invariant 15).
`push_table_update_at_head` → `push_table_update(target_version: Option<u64>)`:
roll-forward pins the classified version; roll-back keeps `None` (publishes the
restore commit it just made). In-scope behavior is preserved, so the existing
roll-forward integration tests are the guard; the drift-hardening is
correct-by-construction (deterministic mid-sweep drift injection isn't feasible
— a sync failpoint can't do an async Lance write).
The repo is a 2-person team where both maintainers own every path, so the
CODEOWNERS machinery (generated CODEOWNERS, roles yml, render script, the two
drift/hand-edit CI jobs) gated nothing real while adding friction: every PR
showed "Review required" and own-PRs merged only via admin/bypass override.
Remove the whole chassis and drop the review gate:
- delete .github/CODEOWNERS, codeowners-roles.yml, render-codeowners.py,
the CODEOWNERS workflow, and docs/dev/codeowners.md
- branch-protection.json: drop the two CODEOWNERS required status checks,
set require_code_owner_reviews=false and required_approving_review_count=0
(CI checks are the gate; maintainers merge their own PRs once green)
- scrub CODEOWNERS references from AGENTS.md, docs indexes, branch-protection
and ci docs, GOVERNANCE.md, and CONTRIBUTING.md
The policy change is inert until an admin runs scripts/apply-branch-protection.sh.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the per-graph MCP surface (POST /graphs/{id}/mcp, shipped in the
preceding commits and landing under v0.8.0) and the `.gq` authoring controls
that shape stored-query tools.
- New docs/user/operations/mcp.md: the client-facing guide — transport, tool
catalog (built-ins + stored queries), projection modes, structured output,
authorization (call-authoritative + list-relaxation), Host/Origin policy, the
protocol-version contract.
- docs/user/operations/server.md: the /mcp endpoint + an "MCP surface" section;
docs/user/index.md: a "Connect an MCP agent" pointer.
- docs/user/queries/index.md: an Annotations section — query @description /
@instruction / @mcp(expose, tool_name) and per-parameter @description.
- AGENTS.md: topic-table row + MCP note on the HTTP-server capability row.
- docs/dev/testing.md: the omnigraph-mcp crate + server tests/mcp.rs.
- docs/dev/rfc-005 §D5: retire the "cluster = everything exposed" bridge —
cluster mode honors source `@mcp(expose: …)`; presentation vs authorization
split made explicit.
- skills/omnigraph: server-policy.md MCP section; stored-queries.md corrected
(per-query controls now ship via @mcp, not "planned"); SKILL.md MCP triggers,
Deep Dives row, version → 0.8.0.
- docs/releases/v0.8.0.md: the MCP surface + authoring-controls release notes.
Crate version manifests are deliberately NOT bumped — that is the v0.8.0
release-cut step; this lands on the feature branch.
Wire the `.gq` authoring surface that controls how a stored query is projected
as an MCP tool. All of it rides in the query source (content-addressed,
re-parsed at boot), so there is no cluster.yaml / catalog / serving-snapshot
plumbing — and it is orthogonal to Cedar `invoke_query` (presentation, not
authorization).
- Per-parameter `@description("…")` (leading the variable) → carried on
`Param.description`, mapped through `param_descriptor`, and emitted on the
outer JSON-Schema property by `param_json_schema`, so it shows up in both the
MCP tool input schema and the `GET /queries` catalog.
- Query `@mcp(expose: <bool>, tool_name: "<name>")` → parsed into
`QueryDecl.mcp`; `StoredQuery::is_exposed()` / `effective_tool_name()` resolve
from it. `expose: false` hides a query from the agent surface (`tools/list`,
`stored_query_list`, run-by-name) while keeping it HTTP/service-callable.
- `@instruction` is folded into the MCP tool description (after `@description`),
so the agent-facing how/when-to-use guidance reaches `tools/list`.
- Removes the now-dead `RegistrySpec.{expose, tool_name}` fields (server + CLI);
`settings.rs` no longer hardcodes `expose: true`. Test helpers express
exposure by injecting `@mcp(expose: false)` into the source (the real path).
openapi.json regenerated: `ParamDescriptor` gains an optional `description`.
Tests: compiler parser (param @description, @mcp parse + duplicate rejection),
api-types schema_equivalence (description on the outer property), server mcp
(folded description + param docs + @mcp tool rename, list==call). Full
workspace gate green.
Add the `omnigraph-mcp` crate (stateless Streamable-HTTP transport, `McpBackend`
seam, fail-closed Host/Origin policy) and the server backend projecting built-in
operations and the per-graph stored-query registry as MCP tools + resources over
`POST /graphs/{id}/mcp`. Every tool delegates to the same engine/handler
functions the REST routes use and is gated by the same Cedar `authorize` path;
reads/writes carry structured output.
Includes three correctness fixes from review + live testing:
- tools/list is a faithful relaxation of the per-call gate: a built-in whose
authorization depends on a caller-chosen branch is shown iff the actor could
invoke it on some branch, via PolicyEngine::permits_on_any_branch (capability
probe through the same Cedar authorizer). A fabricated-`main` probe wrongly
hid graph_mutate under the canonical "protect main, write unprotected" policy.
- The stored-query surface honors mode + `expose` on call as well as on list:
resolve_stored_tool is the single membership test, so the meta pair
(stored_query_list/stored_query_run) is callable only in `meta` mode and
stored_query_run resolves exposed-only. An `expose:false` query is unreachable
by name on the agent surface (it stays HTTP/service-callable).
- The loopback Host allow-list is the full set [127.0.0.1, ::1, localhost]
(matches rmcp's default), so an IPv6 loopback `Host: [::1]` is accepted
regardless of which stack the server bound.
The protocol-version contract is documented (initialize negotiates the version
in its body, so the MCP-Protocol-Version header is validated on non-init
requests only) and pinned by a test.
Tests: omnigraph-mcp/tests/standalone.rs, omnigraph-server/tests/mcp.rs,
omnigraph-policy permits_on_any_branch unit test, omnigraph-api-types schema
projection. Full workspace gate green.
* test(engine): add read-path IO instrumentation seam for warm-read cost tests
Prerequisite seam for the query-latency fixes. Adds
crates/omnigraph/src/instrumentation.rs:
- CountingStorageAdapter: a StorageAdapter decorator counting per-method
reads (read_text/exists/read_text_versioned/list_dir), for the
schema-contract reads on the query path.
- A per-query task-local (QueryIoProbes) carrying Lance WrappingObjectStore
wrappers per open category plus a probe counter, delivered via
with_query_io_probes. open_dataset_tracked attaches the wrapper so the
open itself is counted (ObjectStoreParams.object_store_wrapper).
Wires the wrappers into the manifest open (open_manifest_dataset) and the
commit-graph opens (CommitGraph::open/open_at_branch). Production leaves
the task-local unset, so nothing attaches.
Makes Omnigraph::open_with_storage public so tests can inject the counting
adapter. lance-io is a dev-dependency (IOTracker named only in tests). No
runtime behavior change.
* test(engine): warm same-branch read should reuse the coordinator (red)
Cost-budget test using Lance IOTracker at the object-store boundary (the
LanceDB IO-counted-test pattern). On a 20-commit-deep graph, a warm
same-branch query re-opens a fresh coordinator, which opens both the commit
graph and __manifest. Asserts the read opens the commit graph zero times
and performs exactly one cheap version probe; today it does neither (it
scans the commit graph on re-open and never probes). The freshness guard
already passes. Adds the commit_many helper for history-depth fixtures.
Red half of the Fix 1 red->green pair; turns green with the next commit.
* perf(engine): same-branch reads reuse the warm coordinator (Fix 1)
query()/resolved_target re-opened a fresh GraphCoordinator from storage on
every read (full __manifest scan + two commit-graph scans), so a warm
read's cost grew with commit history (invariant 15) though the data was
unchanged.
resolved_target now serves same-branch reads from the warm in-memory
coordinator, gated by a cheap version probe (latest_version_id, one
object-store op) instead of a full re-open:
- fresh (probe == cached version): return the in-memory snapshot under the
read lock, with a synthetic (branch, version) id and no commit-graph
access (reads pin the snapshot by manifest version, not the commit DAG;
invariant 2).
- stale: take the write lock, re-probe (double-checked; tokio RwLock has no
read->write upgrade), then refresh_manifest_only (no commit-graph scan),
preserving strong consistency for external writers (invariant 6).
Cross-branch and snapshot targets keep the existing cold-resolve path.
Adds ManifestCoordinator/GraphCoordinator::probe_latest_version and
GraphCoordinator::refresh_manifest_only. Nothing on the read path needs a
real commit ULID (only RuntimeCache keys on the id, where synthetic is
consistent), per a caller audit.
A warm same-branch read on a 20-commit graph now does zero commit-graph
opens and exactly one probe (down from a deep commit-graph scan) and still
observes external commits. The residual per-table __manifest scans are
removed later by Fix 2.
* test(engine): warm query should validate the schema contract once (red)
ensure_schema_state_valid runs twice per query (query()/run_query_at AND
resolved_target/snapshot_at_version), each reading 3 contract files + 2
existence probes. A warm query thus does 6 read_text + 4 exists where one
validation (3 + 2) suffices, measured via CountingStorageAdapter. Adds a
drift guard (schema_source_drift_is_caught_on_read) that already passes.
Red half of the finding-A red->green pair.
* perf(engine): validate the schema contract once per query (finding A)
ensure_schema_state_valid ran on every query AND again inside
resolved_target / snapshot_at_version, so each query validated the schema
contract twice (~10 storage ops). Removes the redundant query()/
run_query_at() calls; the validation inside resolved_target /
snapshot_at_version still runs, so drift is detected exactly as before.
A source-only fast path was rejected: a long-lived handle must detect
external drift of the schema source, IR, OR state on its next operation
(lifecycle::long_lived_handle_rejects_schema_*), which a source-only
compare would miss. So the only safe latency win is not validating twice.
A warm query now does one validation (3 read_text + 2 exists) instead of
two (6 + 4).
* test(engine): warm + multi-table reads should do zero manifest scans (red)
After Fix 1 a warm same-branch read still scans __manifest ~44 times at
20-commit depth: not from resolution (Fix 1 removed that) but from the
per-table open path, which routes through the Lance namespace and full-scans
__manifest twice per touched table (describe_table + describe_table_version).
Tightens the warm test to assert manifest read_iops == 0 and adds a
multi-table (traversal) test asserting the same, pinning the "2 tables = 2x"
tax. Red half of the Fix 2 red->green pair.
* perf(engine): open touched tables by location+version, not via the namespace (Fix 2)
SubTableEntry::open routed every read-path table open through
DatasetBuilder::from_namespace(BranchManifestNamespace), whose describe_table
full-scans __manifest and, with managed_versioning, makes Lance scan again
(describe_table_version) -- two full __manifest scans per touched table. That
was the residual that made warm-read manifest IO grow with history and the
'2 tables = 2x' multi-table tax.
The resolved Snapshot already holds each table's path/version/branch, so open
directly: from_uri(table_uri_for_path(root, path, branch)).with_version(v).
The branch-qualified location is the dataset that physically holds the version
(main: {path}; branch: {path}/tree/{branch}, Lance native-branch storage), and
with_version resolves it within THAT dataset's _versions. 0 namespace calls +
1 HEAD via the native ConditionalPutCommitHandler.
The read namespace (BranchManifestNamespace) is now unused in production
(writes use StagedTableNamespace), so it, its constructor, and the helpers only
it used (to_namespace_version, publish_requests, their imports) are gated
#[cfg(test)] -- retained to validate the namespace contract in unit tests.
Removes the dead open_table_at_version_from_manifest.
Warm same-branch + multi-table reads now scan __manifest zero times; branch +
time-travel reads stay correct (branching.rs, point_in_time.rs, 2 lib
regression tests); production-lib warnings unchanged (baseline).
* test(engine): cost-budget coverage for branch-warm and stale-refresh reads (matrix)
Extends the read-path cost-budget tests across more of the morphological matrix:
- warm_branch_read_does_no_manifest_scans: a warm read on a non-main branch
(handle synced to it) scans __manifest zero times, exercising Fix 2's
branch-owned-table open (tree/{branch} + with_version) on Fix 1's warm path --
the cell that regressed when the open used with_branch against the base.
- stale_read_refreshes_manifest_only: an external commit makes the next read
take the stale path, which re-reads the manifest (read_iops > 0) but never
scans the commit graph (refresh_manifest_only), pinning Fix 1's manifest-only
refresh.
Cold paths (cross-branch, time-travel) stay behavior-covered (branching.rs,
point_in_time.rs) and are cold by design (Fix 1 warm-paths only same-branch), so
there is no manifest==0 contract to assert there.
* test(engine): same-branch write after external commit must not fork the commit DAG (red)
* fix(engine): refresh commit-graph head before append to prevent same-branch DAG fork
A same-branch write that follows an external commit committed a fresh manifest
version (commit_all rebases the pin from a fresh coordinator) but appended off
the coordinator's stale in-memory commit-graph head, forking the commit DAG (the
new commit and the external commit shared a parent). Pre-existing for non-strict
inserts; widened to strict ops by Fix 1's refresh_manifest_only freshening the
read-time pin. record_graph_commit now refreshes the commit-graph head from
storage before append_commit, so the parent is the true current head.
record_merge_commit is unaffected (it passes explicit parents).
* perf(engine): hold open Dataset handles + share one Session per graph (Fix 3)
A warm same-branch read still re-opened every touched table per query (the
"never warms up" residual after Fix 1+2). A per-graph held-handle cache keyed by
(table_path, branch, version) now serves repeat reads with zero table opens, and
one shared lance::Session per graph warms metadata/index caches across opens.
Validated against LanceDB upstream (rust/lancedb/src/table/dataset.rs
DatasetConsistencyWrapper): hold an Arc<Dataset> and reuse it for 0-IO warm
reads; one Session per connection threaded into opens; writers never serve from
the read cache; time-travel bypasses. One adaptation: omnigraph keys by version
(snapshot-pins-version model) where LanceDB keys per-table+HEAD, reusing the
in-repo GraphIndexCache LRU template.
- ReadCaches (session + TableHandleCache) injected onto live-Branch-read
snapshots in resolved_target; Snapshot::open serves from the cache or opens
once with the session on a miss (via the instrumented open_table_dataset).
- Writes (resolved_branch_target -> open HEAD) and time-travel / Snapshot-id
reads bypass the cache. Version-in-key makes a write a new key (old handle ages
out via LRU); invalidate_all at branch-switch/refresh is hygiene only.
- Cost tests: a 2nd identical warm read does 0 table opens; a write re-opens only
the changed table at its new version.
Full engine suite green.
* test(engine): forbid raw data opens in the read/exec layer (P2 guard)
Extend the forbidden-API guard with Dataset::open / DatasetBuilder::from_uri /
from_namespace so the read/exec layer (exec/, loader/, changes/, db/omnigraph/)
cannot bypass Snapshot::open and the held-handle cache (Fix 3). The instrumented
opener (instrumentation.rs) is allow-listed; two legitimate non-read opens (a
test editing __manifest, Hard-drop version GC) carry sentinels. The
storage/manifest layers stay allow-listed.
Lean P2 scope, per LanceDB-upstream + minimize-liability: the data-read boundary
already exists (SubTableEntry::open); this guard pins it so a future read cannot
open around the cache. Centralizing all internal opens behind one opener is
deferred.
* docs(dev): invariant 15 (one source of truth, cheaply derived) + cost-budget testing
Records the principle behind the query-latency work: Lance and the manifest are
the source of truth, everything else a derived view held warm and refreshed by a
cheap probe; the two failure modes (a drifting parallel copy, and cold
re-derivation whose cost grows with history) are deny-listed. Adds the
cost-budget testing discipline (assert a warm read's open/IO count is flat at
commit-history depth, the LanceDB IO-counted pattern) and the warm_read_cost.rs
row. Updates the read-path-re-derivation known gap to reflect what Fix 1/2/3 +
finding A close, and adds the commit-graph-parent-under-concurrency gap.
* fix(engine): branch-incarnation identity + unified invalidation + shared LruMap (PR #268 review)
Phase 6 A-D, correct-by-design responses to the Codex/Greptile P2 review comments. A: warm-read freshness and the table-handle cache key use the manifest incarnation (e_tag, manifest-timestamp fallback, then version), so a deleted+recreated non-main branch reusing a version number cannot be served stale; main stays version-cheap, non-main loads latest_manifest; a detected stale refresh also invalidates read caches; two regression tests force the version collision. B: unify the two cache invalidations into Omnigraph::invalidate_read_caches() at the four sites. C: assert the stale path's probe count. D: shared LruMap behind both caches with unconditional eviction, plus a unit test. Full engine suite green; multi-process lineage fork and O(history) write refresh remain known gaps for Phase 6E/7.
* docs: audit pass — drop pre-0.7.0 release notes; scrub RFC refs from user docs
- Delete the pre-0.7.0 release-notes archive (v0.2.0 … v0.6.2); keep v0.7.0.
- Rewrite every inline "RFC-0NN" citation in docs/user/** into durable
plain language (the behavior is the contract, not the planning doc):
cli/index.md, cli/reference.md, clusters/index.md, operations/{maintenance,
policy,server}.md. Updated the in-page "Scopes & profiles" anchor to match
the de-RFC'd heading.
No sub-0.7.0 version caveats or stale Lance-version refs were present in
docs/user/**. Dev docs, AGENTS.md, and instruction files are out of scope for
this pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: second alignment pass — drop residual pre-cluster-only framing
- cli/reference.md: rewrite the server-scope graph-resolution rule — an
omnigraph-server is always cluster-backed, so GET /graphs always answers and
--graph is required; the bare-URL path is only the fallback for an
unavailable/non-omnigraph endpoint (was "a single-graph / flat server …
uses its bare URL as before").
- embeddings.md: "Direct single-graph serving" → "Direct (--store) access"
(there is no single-graph serving mode under cluster-only).
- clusters/{config,index}.md: drop the removed --target flag from the
"--cluster cannot combine with …" clauses.
Verified: no Linear tickets, no RFC refs, no single-graph-as-current, no
--target-as-combinable in docs/user/**.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs(readme): drop em-dashes, Cursor→Codex, rename agent section
- Replace all 20 em-dashes with context-appropriate punctuation (colons,
semicolons, parens, commas) — removes the AI-slop tell.
- Cursor → Codex (the agent-host examples and the MCP host list).
- "Drive it with an AI agent" → "Set it up with an AI agent".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(readme): wordmark header + simplify query examples
- Add the compact wordmark header (light/dark SVG, subtitle, nav row, restyled
badges) from the header-redesign work; bring the wordmark assets with it.
- Rewrite the Query and mutate examples to lead with the short, config-default
form (no repeated --server/--graph) and aliases — showing how simple it is,
not crazy-long lines. The verbose --server/--graph/--store form is demoted to
a one-line "ad-hoc target" note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Adjacent source lines collapse into run-on text when rendered. Add hard line
breaks so the headline, subhead, and each intro sentence land on their own line.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs(readme): server-first rewrite — deploy, agents, on-prem RustFS
Reframe the README around the actual product: a self-hostable, multigraph
graph server for context assembly and multi-agent coordination, deployed
Terraform-style on your own object storage (on-prem via RustFS, or S3/R2/GCS).
- Lead with key capabilities and what you can build, not a local toy.
- Promote "Drive it with an AI agent" (skill + a docs-first setup prompt) above
the manual deploy walkthrough — agents are the primary operator.
- "Deploy" is the hero: cluster.yaml → object store → validate/plan/apply →
omnigraph-server, with RustFS as the on-prem path front and center.
- "Query and mutate": stored queries by name + branch/review/merge.
- Security & governance as scannable bullets; Clients & SDKs as a table.
- Embedded local graph demoted to a clearly-labeled "quick test" (Signal →
Indicates → Pattern), explicitly dev/experiment-only.
- Drop the "serve/served/serving" vocabulary tic in favor of deploy/run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(readme): add the server boot command to Deploy §3 (Greptile P1)
The "Converge and run" step showed only the converge half — the code block
ended at `cluster apply` with no `omnigraph-server` command, leaving a linear
reader without a way to actually start the server. Add the boot line.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(readme): simplify the server boot command
Drop the inline OMNIGRAPH_SERVER_BEARER_TOKEN prefix from the Deploy hero —
the example cluster declares a policy so the server boots without it, and
bearer auth is covered in Security & governance. Leaves a single clean line.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(readme): boot the server from the cluster dir, not a raw s3:// URI
Pointing --cluster at the bucket hardcodes the storage URI in the run command.
Boot from the config directory instead; the storage URI lives once in
cluster.yaml and the server resolves it — single source of truth, and
consistent with the cluster apply commands above.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The substrate makes read/write scaling asymmetric: reads are object-store-backed,
snapshot-isolated, stateless -> horizontal to N replicas with zero coordination;
writes are serialized per (table,branch) by one-winner manifest CAS -> scale by
partitioning (branches/graphs/cells), with a single active coordinator per
(cell,graph,branch). Adds the CQRS deployment split (read fleet / write tier /
maintenance / heavy reads), read-your-write via commit_id/snapshot_id pinning,
and gates pooled WRITES (not reads) on closing the cross-process-CAS gap.
General server/topology/auth/deployment RFC resolving the half-built tenancy
ambiguity (cluster-only server vs pooled tenant_id scaffolding). Decision:
the cluster is the tenant is the cell — silo the data (own storage/catalog/
policy/tokens), pool the compute (one process : N cells). No row-level pooling
(no engine RLS).
- §5.1 CellRuntime lifts today's per-cluster runtime into a value.
- §5.2/§5.3 AppState holds a CellRegistry; resolve_cell is one new outer
middleware hop before auth; the per-graph + Cedar + MCP stack is unchanged.
- §5.4 per-cell CellAuth (Static | Oidc TokenVerifier); WorkOS org -> cell 1:1
with per-cell OAuth audience (cross-tenant token replay fails on aud).
- §5.5 Cedar stays per-graph/per-cell; default-deny-read becomes safe; no
tenant dimension needed.
- §5.6 control plane = Cell Registry (metadata only) + provisioning-as-code;
cell hot-load is the one safe runtime mutation (cell-granular, not graph).
- §5.7 tiered dedicated/pooled/on-prem on one binary; §7 backward-compatible
(today's single-cluster server = a one-cell map).
MCP (rfc-003) is one consumer, not the driver. Linked from docs/dev/index.md.
An external review pass raised 8 findings; verified 7 valid (2 confirmed
against the engine coercer). Folded them in as class-closing fixes rather than
point patches:
- §9.1 (③④, the headline): the JSON-Schema generator was a second hand-written
copy of the engine's input contract — Blob (base64 vs URI string) and nullable
(explicit null) were two drifts of one class. Move the projection to a single
param_json_schema in omnigraph-api-types (next to ParamKind/ParamDescriptor),
fix Blob -> {"type":"string","format":"uri"} (query_input.rs:449 / api-types:354
say blob-URI string) and nullable -> anyOf[..,null] (query_input.rs:273,296),
and lock it to json_value_to_literal_typed with a schema/engine equivalence
test so any future drift is a CI failure.
- §7/§4 (①): replace the fail-open "empty allowed_origins => skip" with a total
OriginPolicy and a single McpHostPolicy::from_bind constructor (remote default
DenyBrowsers, enforced by origin_guard independent of rmcp's empty-list quirk).
No absent-=>-skip state can be constructed.
- §6/§12/§16 (②): make the non-paginated list seam a stated contract (Vec<T>,
no nextCursor; meta mode bounds large catalogs) and drop the pagination claims
the signature couldn't express.
- §9.3 (⑦): built-in/stored tool-name collision becomes a cluster validate/boot
error (fold built-in names into the registry uniqueness check), not a silent
skip — per the invariants deny-list.
- §9.2 (⑥): stored_query_mode folded into the one per-graph mcp: block (Phase 6),
not a floating key; not configurable until that surface exists.
- §10/§1 (⑧): scope derives from the per-graph mount; server-scoped `health`
becomes graph-scoped `graph_health` (server liveness stays REST /healthz).
- §13 (⑤, doc-only): OpenAI row corrected to the `authorization` field; Phase-1
reachability via static bearer is unchanged.
§17 records the locked decisions; the validation header notes the review pass.
Re-validated RFC-003 against the merged tree (RFC-009/010/011/012). Folds in
the deltas that landed on main since the RFC was written:
- §15 routing: RFC-011 made the server cluster-only. GraphRouting::Single/Multi
and the `match state.routing()` branch are gone; build_app always nests
per_graph_protected under /graphs/{graph_id}. So /mcp is always
/graphs/{id}/mcp (even a single-graph boot is a one-graph registry keyed by
`default`). §15.1's per-graph model is now the only model. Line refs repointed
to lib.rs:876/929/953/954/148.
- §9 stored queries: param/catalog DTOs moved to omnigraph-api-types (RFC-009),
re-exported via api.rs — citations repointed (lib.rs:355/373/496). The legacy
omnigraph.yaml `queries:` declaration source is removed; cluster.yaml
graphs.<id>.queries is the sole source. Flag the §D5 bridge: cluster boot
forces expose=true/tool_name=None today, so per-query expose/tool_name is a
planned phase (added to §17 deferred).
- §11 load: /ingest -> canonical POST /load (RFC-009 Phase 5); graph_load reuses
run_ingest via server_load (handlers.rs:1320), /ingest is a deprecated alias.
- Mechanical: policy/handler/identity/ApiError line-number drift repointed
(policy:16/251, handlers:313/334/711/645/913, identity:186, lib.rs:280).
No architectural change — RFC-011 confirms the per-graph model; the rest is
re-citation and the expose/tool_name caveat.
The CI badge read "failing" largely from concurrency-cancelled runs (rapid
doc pushes cancel in-flight runs → cancelled conclusion) and a now-fixed
S3-job break, not from a broken default branch. It's been more noise than
signal; drop it. CI status is still visible on the Actions tab.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs(readme): add embedded quick-start run-through + trim Clients
- Quick start: a copy-pasteable embedded (local file) run-through
(schema → init → load → query → branch), followed by a "Storage backends"
table that surfaces the one thing that changes across embedded / S3 object
storage / RustFS-MinIO — the graph address — with a pointer to cluster mode
for served multi-graph deployments.
- Clients: collapse to a two-line pointer (npm packages + omnigraph-ts repo);
Python SDK marked coming soon.
* docs(readme,quickstart): fix init addressing + drop non-parsing schema commas
Addresses PR review (both verified against source):
- README "Storage backends": `init` takes the graph address as a positional
argument, not `--store` (`Command::Init { uri: String }`) — Codex. Table now
shows bare addresses and a note on which flag each verb uses.
- docs/user/quickstart.md: drop trailing commas in the schema. The .pg grammar
(`prop_decl = { ident ~ ":" ~ type_ref ~ annotation* }`, node body
`(prop_decl | body_constraint)*`, comma not in WHITESPACE) has no comma rule,
so `name: String,` fails to parse — Greptile.
* docs: optimize README for dev onboarding; fix 0.7.0 staleness
The README's setup half drifted from the shipped 0.7.0 CLI and led with the
heaviest path (Docker + RustFS). This reworks it for fast, correct onboarding:
README.md
- New zero-dependency "Your first graph in 60 seconds" hero: a fully
copy-pasteable local file-backed loop (schema → init → load → query → branch).
- Add a correct "Serve it" section (cluster apply + omnigraph-server --cluster);
the server is cluster-only on main, so the old positional-URI boot is gone.
- Demote the RustFS bootstrap to "rehearse the S3 path locally"; reframe the
storage bullet as "filesystem or any S3-compatible store (AWS S3, R2, MinIO,
RustFS)" — RustFS is a provider, not a storage class.
- Fix crate/MCP descriptions (query/mutate/load, not read/change/ingest).
docs/user/quickstart.md
- Fix the query example: `read --name <q> … <uri>` is removed — the query name
is positional and the graph is addressed with `--store` (`omnigraph query
find_people --query queries.gq --store graph.omni`).
scripts/local-rustfs-bootstrap.sh
- Convert to cluster mode: write a cluster.yaml (storage: s3://…), then
validate → import → apply, load the fixture into the derived root with the
now-required --mode, and serve with `omnigraph-server --cluster`. The old
flow (`load` without --mode, `omnigraph-server <URI>` positional boot) no
longer works on a cluster-only server.
* docs: move agent skill into the repo, add agent-setup snippet, drop rustfs script
skills/omnigraph
- The operational skill (formerly `omnigraph-best-practices` in the cookbooks
repo) now lives with the engine it documents, co-versioned. Renamed to
`omnigraph`; repository metadata repointed here.
- Broadened the description to trigger on intent — storing/retrieving/querying
knowledge, agent memory, building a knowledge graph, operating Omnigraph — as
well as on CLI/artifact sightings (stays ≤1024 chars).
- Install: `npx skills add ModernRelay/omnigraph@omnigraph`.
README
- New "Set it up with an AI agent" paste snippet: installs the skill, reads the
docs (URL), browses the cookbooks, and asks the user about a use case before
standing up a first graph.
- "Agent skill & starter graphs" section points at skills/omnigraph + cookbooks.
Drop scripts/local-rustfs-bootstrap.sh
- Not CI-tested (so it rotted: it broke on the cluster-only migration — positional
server boot, load without --mode), demoed the now-optional S3 path, and was the
most fragile artifact in the repo. Replaced with a "Testing against S3 locally"
guide in deployment.md (docker run RustFS/MinIO + AWS_* env + cluster-on-S3).
README/AGENTS references updated.
The RustFS S3 integration job was red on `local_cli_s3_end_to_end_init_load_read_flow`:
the test runs `load --mode overwrite` against an `s3://` target, which the
RFC-011 Decision 9 destructive-write guard now refuses without `--yes`
("refusing destructive `load --mode overwrite` against non-local target …").
The guard is intended and already covered in cli_data.rs; this test slipped
through because it only runs in the bucket-gated RustFS CI job, not the default
local gate. Add `--yes` to match the established pattern used by the other
overwrite-against-non-local tests in this file (lines ~1305, ~1331).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Two accuracy nits from the #263 bot review, against the cookbook's authoritative
addressing reference: `--server` accepts a name OR a literal `http(s)://` URL
(prose had narrowed it to `<name>`), and `cluster apply` should show `--as
<actor>` so the control-plane action's attribution is explicit.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to #262: the command examples led with single-file `--store ./graph.omni`
for everything (init/load/query/mutate/branch), which reads as a single-graph-file
product — the opposite of the cluster-first paradigm.
Reframe so the everyday loop is the headline: declare a cluster → `cluster apply`
→ `omnigraph-server --cluster …` → work against the served graph with
`--server <name> --graph <id>`, invoking stored queries by name. `--store` is
demoted to a clearly-secondary "Local / ad-hoc" note for standalone-graph
iteration. Folds the former separate "Serving" section into step 1.
All served examples verified to resolve correctly (`→ http://…/graphs/<id>
(served)`).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs(readme): align with the cluster-first paradigm + RFC-011 CLI ergonomics
The README's command examples and crate list predated 0.7.0. Update them:
- Common Commands: capability/addressing model — positional/`--store` for direct
storage, `--server` for served graphs (no positional `http(s)://`), `query`/
`mutate` invoke a stored query by name (positional is the query name), `load`
requires `--mode`. Drop the false "same URI works for local/s3/http" claim and
the deprecated `read`/`change` + `--name` forms; mention `~/.omnigraph/config.yaml`.
- New "Serving (cluster-first)" section: a deployment is a cluster.yaml converged
with `cluster apply` and served by `omnigraph-server --cluster <dir|s3://…>`
(no single-graph mode; config-free boot from a bucket).
- Fix the stale docs link (`docs/user/cli.md` → `cli/index.md` + the CLI reference)
after the docs were restructured into topic sections.
- Workspace Crates: list all seven (add omnigraph-policy, omnigraph-api-types,
omnigraph-cluster) with cluster-first framing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(agents): fix the stale `read --name` quick-reference example
AGENTS.md (always-loaded; symlinked to CLAUDE.md) still showed the deprecated
`omnigraph read --query … --name … <s3-uri>` form. Update it to the RFC-011
shape: `omnigraph query --query … <name> … --store <uri>` (read→query, --name→
positional query name, positional URI→--store). Addresses the bot-review finding
on #262.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The release matrix ran three jobs (linux/macos/windows) that each called
`softprops/action-gh-release` to create-or-update the SAME release. Concurrent
"Finalizing release" calls exhausted the action's retries, so whole platforms'
assets were dropped (on v0.7.0: linux won; macOS and Windows failed and their
binaries never attached).
Split build from publish:
- the matrix now only uploads workflow artifacts (`actions/upload-artifact`);
- a single `publish_release` job downloads them all and makes one
`action-gh-release` call — the sole writer, so no race.
Also add a `tag` workflow_dispatch input (resolved as `inputs.tag ||
github.ref_name`) so a tag can be re-published without re-cutting it — used to
finish v0.7.0. `update_homebrew_tap` and `smoke_windows_installer` now depend on
`publish_release` and use the resolved tag (not `GITHUB_REF_NAME`, which is the
branch on a dispatch).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The publish-crates workflow's list predated two workspace crates: the shared
wire-DTO crate `omnigraph-api-types` (RFC-009) and `omnigraph-cluster`. On the
v0.7.0 tag it published compiler/policy/engine, then failed on `omnigraph-server`
("no matching package named `omnigraph-api-types`") because that dependency was
never published.
Insert both in dependency order (after omnigraph-engine, before
omnigraph-server). The workflow is idempotent (per-crate version check), so a
re-dispatch for v0.7.0 skips the three already-published crates and finishes
api-types → cluster → server → cli.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Reconcile the v0.7.0 release notes with what 0.7.0 actually ships. The draft was
mid-cycle; two facts changed and a late-cycle arc was missing.
- omnigraph.yaml is REMOVED (not deprecated): drop the deprecation-window
framing (config migrate, OMNIGRAPH_NO_LEGACY_CONFIG, OMNIGRAPH_SUPPRESS_YAML_
DEPRECATION); the two-surface config (cluster.yaml + ~/.omnigraph/config.yaml)
is the only config.
- Cluster-only server: the server boots only from --cluster; no single-graph
flat-route / positional-URI / omnigraph.yaml-graphs boot. Deprecated-route Link
headers are the sibling-relative form (<load>, not </load>).
- Add the RFC-011 tail: defaults.store, profile list/show, schema-apply refusal
(CLI signpost + server 409), read-only aliases, the any/served/direct/control/
local capability vocabulary, removed legacy data-plane addressing.
- New "Engine & substrate" section: Lance 6→7, indexed traversal, scalar-index/
query-latency, index-materialization-deferred, recovery liveness, branch-fork
self-heal, composite @unique.
- New "Embeddings (RFC-012)" section + breaking bullet: provider-independent
client (OpenRouter default), @embed same-space validation, the default-provider
flip (OMNIGRAPH_EMBED_PROVIDER=gemini for Gemini-direct; OMNIGRAPH_GEMINI_BASE_URL
dropped).
- Upgrade notes: replace the false "omnigraph.yaml keeps working / config migrate"
guidance with the manual cluster.yaml + operator-config path; add server
--cluster and embeddings notes.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix rfc-011 follow-up regressions
* test(cli): remove served schema-apply tests obsoleted by the cluster 409
This PR disables server-side schema apply for cluster-backed serving (409 →
`omnigraph cluster apply`). Two system_local tests still drove *served* schema
apply against a spawned `--cluster` server and asserted the pre-409 behavior, so
they failed under `cargo test --workspace`:
- `local_cli_schema_apply_enforces_engine_layer_policy` — expected a per-actor
policy `denied`/allow on the served route; the route now 409s for everyone
before policy runs.
- `local_cli_schema_apply_rejects_stored_query_breakage_before_publish` —
expected a served apply to reject a stored-query breakage; the route now 409s
before any apply.
Both exercise a path the PR intentionally removed. Their surviving coverage:
the 409 itself is pinned by `schema_routes::schema_apply_route_refuses_cluster_backed_server_mode`
(asserts 409 + no mutation); stored-query-breakage-before-publish stays covered
by `schema_routes::schema_apply_route_rejects_stored_query_breakage_before_publish`
(single-mode); engine-layer schema_apply Cedar enforcement stays covered by
`policy_engine_chassis`. Remove the obsolete served versions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(server): report the cluster-backed schema-apply 409 after the Cedar gate
The 409 ("schema apply is disabled for cluster-backed serving") fired at the top
of `server_schema_apply`, before `authorize_request`. An authenticated-but-
unauthorized actor therefore learned the server is cluster-backed (409) instead
of getting a normal 403 — leaking topology before authorization, against the
same posture that keeps `GET /graphs` default-deny.
Move the 409 below the Cedar gate so the route reports 401 → 403 → 409: an
unauthorized actor gets 403, and only an actor authorized for `schema_apply`
sees the actionable "use `omnigraph cluster apply`" 409. (An open/unauthenticated
server still 409s, as it has no topology to protect.)
Regression: `schema_apply_route_cluster_backed_denies_unauthorized_actor_before_409`
(POLICY_YAML grants no schema_apply → act-ragnor gets 403, not 409). Addresses the
bot-review finding on #258.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
`render_annotations` emitted `@embed` values unquoted — `@embed(title,
model=openai/text-embedding-3-large)`. The parser stores values via
`decode_string_literal` (quotes stripped) and `annotation_kwarg` requires a
quoted `literal`, so the rendered output did not re-parse: a `model` containing
`/` or `-` is not a valid bare token. `schema show` therefore produced schema
text that `schema apply`/lint would reject.
Re-quote the positional value and every kwarg value as string literals, so
`schema show` reproduces `@embed("title", model="openai/text-embedding-3-large")`
and round-trips. Regression: `render_annotations_quotes_values_so_embed_round_trips`
parses the rendered form back through the schema grammar.
Addresses the bot-review finding on #248.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`EmbeddingConfig::from_parts(Some("mock"), _, Some(model), _)` short-circuited to
`Self::mock()` and silently dropped the provided `model`, falling back to
`OMNIGRAPH_EMBED_MODEL`. A cluster `embeddings` profile that sets
`provider: mock, model: X` (RFC-012 Phase 5) would therefore not resolve to X —
the query-time same-space check (`exec/query.rs`) compares the recorded model
against the resolved one, so a dropped model makes that check fire on the wrong
value (a spurious mismatch, or env-dependent). Mock vectors are model-independent
so this is not silent cross-space ranking, but it breaks the contract `from_parts`
documents ("model defaults exactly as from_env does") and the cluster path that
Phase 5 wiring will feed it.
Honor an explicit `model` for mock; fall back to `mock()`'s env-based model only
when none is supplied. Regression: `from_parts_mock_honors_an_explicit_model`
(red before this change).
Addresses the bot-review finding on #248.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The `local` line of the `--help` "COMMANDS BY CAPABILITY" legend was stale: it
still listed `config` (the `config migrate` group was removed with the
omnigraph.yaml excision) and omitted `profile` (added by RFC-011 D8). Update the
list to `embed, login, logout, profile, version`.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Inspect the per-operator `~/.omnigraph/config.yaml` scope profiles without
running anything:
- `profile list [--json]` — every profile with its binding (server/cluster/store)
and default graph; marks the `$OMNIGRAPH_PROFILE`-active one. A malformed
(zero/two-scope) profile is reported as `invalid: <reason>`, not a hard failure.
- `profile show [<name>] [--json]` — one profile's resolved scope: binding kind +
target, the resolved endpoint (a server's URL / a cluster's root / the store
URI), default graph, and output format. With no name, shows the active
(`$OMNIGRAPH_PROFILE`) profile, else the flat operator defaults.
Both are `local` (Session plane) — they read operator config only, take no
addressing flags. Display reads `OperatorProfile::binding()` + the same
`servers`/`clusters` lookups the scope resolver uses (not `resolve_scope`, which
is capability-gated and can't render all three binding kinds at once), so it is
honest about what a profile binds.
Also: RFC-011 bookkeeping (Status → Accepted; D8 shipped, D11 gated on RFC #219,
D5 deferred) and drop the stale "legacy config actor (RFC-008 window)" comment in
operator.rs (the legacy actor is gone).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* chore: correct stale global-lock comments
The global Arc<RwLock<Omnigraph>> that once serialized every server write was
removed — the server holds the engine as a lockless Arc<Omnigraph> and write
methods are &self, so the per-(table_key, branch) write queues are now the
actual write-serialization mechanism (in-process only).
Correct comments that still claimed the global lock is 'still in place' /
'today', or framed the queues as MR-686 scaffolding: write_queue.rs module doc,
exec/merge.rs, db/omnigraph/schema_apply.rs, db/manifest/recovery.rs, and the
bench_concurrent_http.rs example (which also wrongly stated mutate_as is
&mut self). workload.rs is left as-is — its 'previous global RwLock' wording is
accurate history.
* test: regression for self-healing a manifest-unreferenced fork
An interrupted first-write fork (create_branch succeeded, the manifest publish
did not) leaves a fully-formed Lance branch ref the manifest never references.
The branch stays a valid manifest branch, so cleanup's reconciler never
reclaims it, and today the next write to that table wedges with 'incomplete
prior delete; run cleanup'.
Forge that exact residue (a live 'feature' branch + a directly-created
'feature' ref on the Person table the manifest doesn't reference) and assert
the next load AND mutate self-heal. Deterministic and local — no S3 or timing,
since the forge IS the post-crash state. Adds a shared node_table_uri helper.
This commit is RED: it reproduces the bug and fails against the unfixed engine
with the predicted symptom. The fix follows in the next commit.
* fix: self-heal manifest-unreferenced branch forks
The first write to a table on a branch lazily forks it via Lance create_branch,
a durable two-phase op that advances Lance state BEFORE the atomic manifest
publish. If the writer dies or its request future is cancelled between the fork
and the publish, the branch ref is fully formed but the manifest never
references it. The next write re-enters the fork path, create_branch collides,
and the engine wedged with 'orphaned table state ... incomplete prior delete;
run cleanup' — which cleanup could not even fix, because the branch is still a
live manifest branch. This hit load, mutate, ingest, and the merge fork path
(one shared engine chokepoint), so a routine deploy restart or client
disconnect could wedge a branch.
Fix: treat the per-table fork ref as derived state of the manifest. fork_branch_
from_state returns a typed ForkOutcome instead of a human 'incomplete prior
delete' error; on RefAlreadyExists the db layer reclaims the manifest-
unreferenced fork (force_delete_branch + re-fork, exactly once) and proceeds.
A live committed fork is still routed to a retryable conflict before the fork
path, so concurrent first-writes stay correct.
Reclaim is only safe if no in-process writer can be mid-fork, so the write
entry points (load, mutate) acquire the per-(table, branch) write queues for
all touched tables up front — before the fork, held through the publish — when
forking a non-main branch. commit_all accepts these pre-held guards instead of
re-acquiring (the queue is non-re-entrant). The merge fork path already holds
the queue and self-heals through the shared wrapper. Cross-process in-flight
forks remain the documented one-winner-CAS gap.
Mechanical prep folded in: mutation IR lowering is hoisted so the touched-table
set is known before execution; commit_all gains the held_guards parameter.
Flips recreate_over_orphaned_fork_before_cleanup_is_actionable to assert
self-heal; fork_collision_with_live_concurrent_fork_is_retryable still holds.
Docs: writes.md cancelled-future note, invariants.md cross-process known gap.
* fix(cleanup): reconcile per-table manifest-unreferenced forks
reconcile_orphaned_branches keyed orphans on the branch NAME (absent from the
manifest), so it only reclaimed forks from a fully-deleted branch. A fork left
on a still-live branch by an interrupted first-write was never reclaimed — the
backstop the handoff expected cleanup to provide did not cover that case.
Broaden it to a per-table authority test: a Lance branch B on table T is an
orphan iff B is not a live manifest branch (delete-leftover) OR the manifest's
branch-B snapshot does not place T on B (interrupted first-write). Per-branch
snapshots are resolved once and cached across tables. Legitimately-forked
tables, main, and internal/system branches are never reclaimed; children are
dropped before parents to avoid Lance's referenced-parent RefConflict. The
commit-graph half stays whole-branch (per-table doesn't apply there).
This is the guaranteed-convergence backstop to the write-path self-heal: it
reclaims any fork the write path never revisits, and is what Lance's own
create_branch docstring asks embedders to provide for zombie/orphan refs.
* fix: reclaim self-validates against fresh manifest authority
The fork reclaim force-deletes a Lance branch ref, gated on the caller's proof
that the manifest does not place the table on the branch. But the first-write
path obtains that proof via snapshot_for_branch, which returns the coordinator's
CACHED snapshot when the handle is bound to the branch (an embedded handle on
the branch, or branch_merge's target swap). If that snapshot is stale and a
concurrent writer already published a legitimate fork, the reclaim would
force-delete it and re-fork from source, stranding the manifest at a version the
recreated ref no longer has.
Make the destructive primitive own its safety precondition: re-derive it from a
FRESH manifest read (fresh_snapshot_for_branch, which bypasses the cache)
immediately before force-deleting. If fresh authority shows the table is on the
branch, refuse with a retryable conflict instead of destroying a valid fork.
Correct for any caller regardless of snapshot staleness. Also stop branching on
Lance's exact RefConflict prose (loosened match; typed-variant is the durable
follow-up). Addresses PR review (Codex P1, Greptile P2).
* fix: cover delete-cascade edges in up-front fork-queue acquisition
A node delete cascades to every edge table touching that node (execute_delete_
node), forking those edge tables during execution. But touched_table_keys
derived the up-front fork-queue set from the IR ops alone (just node:Type), so a
branch delete that forks node + cascade edges held only the node queue —
commit_all then saw cascade-edge keys it had no guard for.
The touched set is a pure function of (IR ops + catalog), so compute the
COMPLETE set: op types plus, for delete-node ops, the cascade edges derived the
same way the executor derives them (from_type/to_type match). Pre-computed now
equals actual by construction.
Also promote commit_all's held-guard coverage check out of debug_assert into an
all-builds check that fails the write with a typed manifest_internal error: a
load-bearing serialization invariant must fail loudly+safely in release, not
silently proceed unguarded if a future execution path ever touches a table
outside the pre-computed set.
Adds branch_cascade_delete_forks_node_and_edges_under_held_queues, which drives
the cascade path on a branch (the gap the existing insert/load tests missed).
Addresses PR review (Cursor medium, Greptile P2).
* fix(cleanup): serialize fork reclaim against in-process live writers
The broadened per-table reconciler force_delete'd an orphan candidate on a LIVE
branch without holding the per-(table, branch) write queue. An in-process
first-write fork in its fork->publish window holds that queue and has not yet
advanced the manifest, so it looks exactly like an origin-2 orphan — concurrent
cleanup could delete the ref the writer still holds and is about to publish.
(The old branch-name-based reconciler did not have this race: a deleted branch
cannot have a live first-write.)
Bring the reconciler under the same invariant the write-path reclaim already
obeys: never force_delete a fork ref without holding the (table, branch) write
queue AND confirming, under it, from a fresh read, that the ref is still
manifest-unreferenced. Acquire one key at a time (no lock-order inversion vs
multi-table acquire_many writers); if the writer published meanwhile, the fresh
re-check sees the table on the branch and skips. Cross-process writers remain
the documented one-winner-CAS gap. Addresses PR review (Cursor high).
* fix: classify create_branch failure by ref existence, not by failure
fork_branch_from_state mapped ANY create_branch failure to RefAlreadyExists,
routing transient I/O / version / Lance-internal errors into the destructive
reclaim path and masking the real error as a retryable conflict.
Branch on the actual fact instead: on create_branch failure, check whether the
ref exists (list_branches). Only a genuinely pre-existing ref — a fully-formed
manifest-unreferenced fork — is a reclaim candidate; any other failure
propagates with fidelity. We deliberately do NOT force-delete on a not-found-ref
failure: it is indistinguishable from a transient error on a fresh create, and
force-deleting there is the overreach the fresh-authority guard already removed.
A phase-1-only Lance zombie (rarer; create_branch interrupted mid its two
internal phases) surfaces as the propagated error for manual reclaim.
Addresses PR review (Cursor medium).
* fix(cleanup): skip (not delete) on a transient re-check error for a live branch
The reconcile pre-delete re-check treated ANY fresh_snapshot error as 'still an
orphan' and proceeded to force_delete. A transient manifest read failure on a
LIVE branch could therefore destroy a fork the manifest still considers
legitimate — inconsistent with the write-path reclaim (aborts on the same error)
and the candidate scan (skips on snapshot failure).
Distinguish the two origins under the queue: a branch absent from the manifest
authority (origin 1) is a confirmed orphan and is deleted without a fresh read
(no live writer can hold a deleted branch's queue); a LIVE branch (origin 2)
gets the fresh re-check and, on a transient read error, is SKIPPED — never
destroyed on ambiguity — converging on a later cleanup. Same don't-destroy-on-
ambiguous-error principle as the create_branch failure classification.
Addresses PR review (Cursor medium).
* fix(cleanup): unify fork-ref reclaim on fresh authority under the queue
Consolidates the reconcile/reclaim hardening from PR review (the earlier per-site
commits were collapsed when reconciling with the main merge). Both destructive
fork-ref sites — the write-path reclaim and the cleanup reconciler — now share
one classifier, classify_fork_ref -> ForkRefStatus { Legitimate, Orphan,
Indeterminate }, evaluated from FRESH manifest authority under the held
(table, branch) write queue. A fork ref is destroyed ONLY on a confirmed Orphan;
a Legitimate (concurrent writer published a real fork) or Indeterminate
(transient read) status is never destroyed — the write path maps it to a
retryable conflict, cleanup maps it to skip. This closes, by construction:
- reclaim trusting a possibly-cached caller proof (Codex P1);
- reconcile racing an in-process live fork without the queue (Cursor);
- delete-on-transient-error in the re-check (Cursor/Greptile);
- origin-1 trusting a stale live_branches capture for a created-since branch
(Cursor/Greptile P1).
Having one classifier removes the duplication that let the two sites drift.
ForkOutcome is made pub to match the sealed trait method returning it. Verified
green on Lance 7.0.0 (full engine suite + 48/48 failpoints).
* test(cleanup): pin classify_fork_ref decision (Legitimate / Orphan / ghost)
Both fork-ref reclaim sites (write-path reclaim + cleanup reconciler) route
their destroy/skip decision through classify_fork_ref, but it had no direct
test — reverting the fresh-authority logic was not test-detectable. Add a
deterministic in-source unit test that forges each state and asserts the status:
a manifest-placed fork -> Legitimate (never destroyed); a ref the manifest does
not place on the branch -> Orphan; a ref for a branch absent from the manifest
-> Orphan (ghost reclaim preserved). This makes the core fresh-authority
decision behind every reclaim fix revert-detectable in one place.
(The Indeterminate arm — transient read on a live branch -> skip — needs an
injected read failure and is left to the failpoints suite; the cross-process
cleanup-vs-writer and cached-snapshot reclaim races are the documented
one-winner-CAS gap, not reachable same-process bugs, so they are not faked here.)
* test(cleanup): pin the Indeterminate (transient re-check) reclaim arm
Closes the last untested classify_fork_ref arm. Adds a 'classify.fresh_read'
failpoint (no-op without the failpoints feature) that simulates a transient
failure of the fresh-authority read, and a failpoints test driving it through
cleanup: a genuine origin-2 orphan on a LIVE branch whose fresh re-check fails
classifies as Indeterminate, so the reconciler SKIPS it (never destroys on an
inconclusive read) and reclaims it on the next run once the read succeeds.
This makes the don't-destroy-on-ambiguity rule revert-detectable end-to-end.
The only paths now left untested are the cross-process cleanup-vs-writer and
reclaim-vs-publish races — the documented one-winner-CAS gap (cleanup is
&mut self / CLI-only, so no reachable same-process race), not faked here.
* test(server): avoid stale schema apply route handle
* fix(cleanup): report indeterminate fork authority clearly
`omnigraph schema apply` against a cluster-managed graph's storage root bypassed
the cluster ledger/recovery/approvals. Mirror `init`'s refusal: on the embedded
(direct-store) path, if the resolved URI is inside a cluster
(`cluster_root_for_graph_uri`), bail and point at `cluster apply`. The served
(`--server`) path is unaffected — it addresses a server, not a storage root.
`schema plan`/`show` (read-only) are untouched.
Two e2e tests injected "out-of-band drift" via this exact CLI path; since the
CLI now refuses it, they inject drift via a direct engine `apply_schema` against
the storage root instead — a faithful control-plane bypass, which is what
out-of-band drift is. New regression:
`schema_apply_refuses_a_cluster_managed_graph_and_signposts_cluster_apply`.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Engine: factor the alias->(provider, base_url, model, key-envs) defaults into a shared provider_profile, used by both from_env and a new pub from_parts(provider, base_url, model, api_key) so the cluster path applies identical defaults (DRY).
Cluster: GraphConfig gains an optional per-graph embeddings: profile (EmbeddingProfile { provider?, base_url?, model?, api_key }). EmbeddingProfile::resolve() resolves the api_key from a ${NAME} env ref (resolve_secret_ref rejects inline values and unset vars) and builds an engine EmbeddingConfig via from_parts.
Parse + resolve + validate only; wiring the profile through the applied cluster state into the served handle (the config-free s3:// boot reads applied state, not cluster.yaml) follows. 21 engine + 4 cluster tests pass.
The Omnigraph handle gains an optional embedding_config (Arc<EmbeddingConfig>) injected via with_embedding_config(), mirroring with_policy(). The query path now threads an EmbeddingResolver (bundling the reuse OnceCell + the optional injected config) instead of the bare cell; its lazy resolve() builds the client from the injected config when present, else EmbeddingClient::from_env(). Laziness preserved (a graph that never embeds needs no key). This is the engine half of cluster per-graph embedding wiring; the cluster config + serving injection follow. New search test proves the injected config is used (from_env would error with no keys).
Self-contained: no cluster dependency. 25 search tests pass.
openai-alias api key (Cursor High): key resolution was dispatched on the Provider enum, which is OpenAiCompatible for both openai and openai-compatible, so OMNIGRAPH_EMBED_PROVIDER=openai (base api.openai.com) could send OPENROUTER_API_KEY and 401. Fix: the alias match now yields the ordered key-env list too, so base, model and key are all alias-derived in one place. openai takes only OPENAI_API_KEY (errors loudly if absent); openai-compatible/unset prefer OPENROUTER_API_KEY then OPENAI_API_KEY. Closes the class, not the instance.
deadline scope (Cursor Medium): the deadline bounds every embed call (query and document), which is correct, but the name said query-only. Renamed OMNIGRAPH_EMBED_QUERY_DEADLINE_MS -> OMNIGRAPH_EMBED_DEADLINE_MS (field query_deadline_ms -> deadline_ms) and updated the doc wording so the name matches the behavior. Two new alias-key tests; 20 embedding unit tests pass.
Resolved earlier in 30377c4: openai-alias base URL, single embed-model source, stale @embed ingest docs. Declined: RFC Status (the lifecycle keeps it Proposed while the PR is open).
* refactor(cli): own ReadOutputFormat/TableCellLayout in the CLI
The two output-presentation enums lived in `omnigraph-server::config` and were
re-exported for the CLI, even though the server never used them. Move both
definitions into `omnigraph-cli/src/read_format.rs` (where the renderer already
lives) and drop them from the server's public re-export. This is a step toward
deleting the legacy `omnigraph-server::config` module entirely — a CLI
presentation concern has no business in the server crate.
No behavior change. The server keeps private copies in `config.rs` only for the
soon-to-be-deleted legacy `CliDefaults`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(cli)!: remove the `config migrate` command and migrate.rs
`config migrate` was the last CLI consumer of the legacy `omnigraph.yaml`
(`OmnigraphConfig` + `load_config`). With the excision complete there is no
legacy file to split, so the whole `omnigraph config` command group is removed
along with `migrate.rs`. The `OmnigraphConfig` type, `load_config`, and the
deprecation machinery are deleted next.
- Remove `Command::Config` / `ConfigCommand` from the clap surface and the
dispatch arm; drop `mod migrate;` and the now-unused `load_config` import.
- Drop the `Command::Config` arms in `planes.rs`.
- Delete the `config_migrate_splits_legacy_config` integration test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(server)!: delete the legacy OmnigraphConfig type and load_config
With `config migrate` gone, nothing loads `omnigraph.yaml` anymore. Delete the
entire `omnigraph-server::config` module: the `OmnigraphConfig` type and its
sub-structs (`ProjectConfig`, `TargetConfig`, `CliDefaults`, `ServerDefaults`,
`AuthDefaults`, `QueryDefaults`, `AliasConfig`, `AliasCommand`, `PolicySettings`,
`QueryEntry`, `McpSettings`), `load_config`, and the RFC-008 deprecation
machinery (`OMNIGRAPH_CONFIG`, `OMNIGRAPH_NO_LEGACY_CONFIG`,
`OMNIGRAPH_SUPPRESS_YAML_DEPRECATION`, the deprecation map + warner).
- `QueryRegistry::load` (the only `OmnigraphConfig`/`QueryEntry` consumer; its
only caller was its own test) is removed — server boot and the CLI both build
registries via `QueryRegistry::from_specs`.
- `graph_resource_id_for_selection` (CLI-only) moves into the CLI
(`helpers.rs`), with its unit test; the server no longer exports it.
- Drop the already-dead `format_registry_load_errors` helper (config-adjacent).
No behavior change — every deleted item was unreachable after the excision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: purge the legacy omnigraph.yaml surface from the docs
Finish the RFC-011 excision in the docs: the CLI no longer reads omnigraph.yaml
and the server boots cluster-only, so every doc that described the legacy file
as a live config is now wrong.
- AGENTS.md: rewrite the HTTP-server line to cluster-only boot (drop the
single-graph/flat-route and omnigraph.yaml-boot framing); rewrite the CLI
two-surface-config passage (drop `config migrate`, the deprecation env vars,
and "Never extend omnigraph.yaml"); fix the topic table + capability rows.
- cli/reference.md: delete the entire "omnigraph.yaml schema (legacy combined
file)" section and the `config migrate` row; re-home the `policy` row, the
bearer-token chain, the actor/format/param-precedence references, and the
`--config` mentions to the operator config + `--cluster`.
- cli/index.md: rewrite the multi-graph-server + add-graph paragraphs to
cluster (`--cluster` + `cluster apply`); fix the policy examples to
`--cluster`; replace the `## Config` omnigraph.yaml example with the
operator/cluster two-surface model.
- operations/policy.md: rewrite per-graph-vs-server-level policy to the cluster
`policies:`/`applies_to` model; re-home the actor + CLI tooling sections.
- clusters/config.md, clusters/index.md, deployment.md: server boots from the
cluster only; per-operator facts come from ~/.omnigraph/config.yaml.
- architecture.md, testing.md: drop the stale omnigraph.yaml / deleted-test
references.
RFCs, design specs, and prior release notes are left as historical records.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
