* refactor(storage): gate test-only TableStore::append_batch behind cfg(test)
The inherent append_batch is used only by in-source recovery test setup, but
the non-test lib build (cfg(test) off) cannot see those callers and emitted a
dead_code warning. Gating the method #[cfg(test)] silences the false positive
and enforces its own doc contract ("no new engine call sites") by construction
— engine code physically cannot call a cfg(test) method.
* test(failpoints): harden fault-injection harness + reproduce roll-forward CAS race
Hardens the test infrastructure around the process-global `fail` registry, and
adds a deterministic red repro for the open-time recovery sweep's roll-forward
CAS race (iss-schema-apply-reopen-recovery-race). The fix lands in the next
commit — this commit is intentionally red (rule 12: red→green visible in log).
Harness:
- One `ScopedFailPoint` (engine) gaining `with_callback`; the cluster duplicate
is removed and cluster tests reuse the engine type via `omnigraph/failpoints`.
- `#[serial]` on every failpoint test (the registry is process-global, so shared
names interfere under parallelism); `serial_test` added to cluster dev-deps.
- `helpers::failpoint::Rendezvous` (park-first / wait-until-reached / release)
replaces fixed-`sleep` cross-thread coordination; the three concurrent tests
now rendezvous deterministically. The reached flag doubles as a fired-assert.
- Compile-checked `failpoints::names` catalog (engine + cluster); every call
site references a const, and `failpoint_names_guard.rs` enforces "no string
literal names" by source-walk, so a typo is a build error not a silent no-fire.
Red repro:
- New `recovery.before_roll_forward_publish` failpoint at the sweep's
classify -> publish-CAS window (the only injection point there).
- `open_sweep_roll_forward_converges_when_manifest_advances_concurrently`: two
concurrent open-sweeps race one pending sidecar; the sweep parked at the
failpoint loses its publish CAS to the other and fails the open with
`ExpectedVersionMismatch`. FAILS at this commit by design.
* fix(recovery): converge roll-forward when the manifest advances concurrently
The open-time recovery sweep classified a pending sidecar as RolledPastExpected,
then published a manifest CAS at the sidecar's pinned expected_version. Under a
concurrent writer that advanced the manifest past expected during the
classify -> publish window, the CAS failed with ExpectedVersionMismatch and
`?`-propagated, failing the whole Omnigraph::open.
iss-schema-apply-reopen-recovery-race.
A roll-forward's postcondition is "the manifest reflects the sidecar's committed
Lance state", not "this sweep won the CAS" (invariants 7 & 15). On an
ExpectedVersionMismatch, re-read the live manifest and check whether the
sidecar's intent is already satisfied (every pinned table at a version >= the
one we observed and tried to publish; added tables registered; tombstones gone
— sound under the heal-first invariant, documented at the check). If satisfied,
this is convergence: record the RolledForward audit + delete the sidecar
idempotently. If only partway, defer to the next pass. Either way the open no
longer fails. Other errors still propagate; a genuine logical conflict
resurfaces via the classifier's InvariantViolation.
Turns the red repro from the previous commit green. The roll-BACK twin
(iss-recovery-sweep-live-writer-rollback) is destructive (Lance Restore) and
still needs a cross-process lease — the known-gap is updated accordingly.
* Address PR review: harden failpoint name guard + dedupe converge audit
Two issues surfaced in PR review of the failpoint hardening + recovery fix:
1. Name guard had a line-split blind spot. It scanned per line, so a call
wrapped across lines (`park_first(\n "name",\n)`) put the literal on a
different line than the call prefix and bypassed the "no string-literal
failpoint names" check — and one such literal
(`mutation.delete_node_pre_primary_delete`) had slipped through. Make the
guard whitespace/newline-tolerant (skip past the open paren to the first
argument token) so wrapping can't hide a literal, and convert the bypassed
site to the `names::` const.
2. Convergence path could append a duplicate recovery audit. When a
roll-forward publish loses its CAS but the manifest already reached the
sidecar's goal, `converge_or_defer_roll_forward` recorded a RolledForward
audit unconditionally. Under the heal-first invariant, whoever advanced the
manifest already healed this sidecar (audit + delete), so a second row
landed in `_graph_commit_recoveries` for one recovery event. Gate the
audit+delete on the sidecar still being present: absent => the winner
completed it, return success with no duplicate row. The convergence
regression test now asserts exactly one audit row.
* docs(dev): remove the schema-apply recovery-flake handoff (fixed by this PR)
The handoff was a transient investigation note for
`iss-schema-apply-reopen-recovery-race`, which this PR fixes (the converge
helper + the red→green regression). Its rationale now lives durably in the
dev-graph issue, the PR/commit history, and invariants.md, so the handoff is
obsolete. Drop the doc, its dev-index row, and the dangling reference from the
RFC-013 handoff; the doc cross-link check stays green.
* fix(recovery): include added-table registrations in the converge audit
The CAS-loss convergence audit built outcomes only from `sidecar.tables`,
omitting the `additional_registrations` that the normal `roll_forward_all`
audit includes. For a SchemaApply sidecar with added types, a converge-path
audit row would be incomplete versus the normal roll-forward path for the same
recovery kind. Mirror the roll-forward outcome construction (append a
registration outcome per added table) so both paths emit the same audit shape.
|
||
|---|---|---|
| .cargo | ||
| .context | ||
| .github | ||
| assets | ||
| crates | ||
| docker | ||
| docs | ||
| scripts | ||
| skills/omnigraph | ||
| .dockerignore | ||
| .gitignore | ||
| AGENTS.md | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CLAUDE.md | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| Dockerfile | ||
| GOVERNANCE.md | ||
| LICENSE | ||
| og-cheet-sheet.md | ||
| omnigraph.example.yaml | ||
| openapi.json | ||
| README.md | ||
| rust-toolchain.toml | ||
| SECURITY.md | ||
Lakehouse graph database for context assembly & multi-agent coordination
Multimodal retrieval · Git-style branching · object-storage native
Quickstart · Docs · Cookbooks · CLI
Omnigraph is the operational state and coordination layer for fleets of agents.
Run it as a server, declared as code; hundreds of agents operate and enrich the graph on parallel isolated branches, and every change is reviewed and merged safely.
Key capabilities
| Capability | What it gives you |
|---|---|
| Declared as code | A cluster.yaml declares graphs, schemas, stored queries, embedding providers, and policies; cluster apply converges it and omnigraph-server brings every graph online at /graphs/{id}/…. |
| Built for fleets of agents | Hundreds of agents enrich the graph on parallel isolated branches; changes are reviewed and merged safely, Git-style, across the whole graph. |
| Multimodal retrieval | Graph traversal + vector ANN + full-text + Reciprocal Rank Fusion in one query runtime, for context assembly. |
| Security as code | Cedar policy enforced server-side on every mutation, per-graph and server-wide; bearer auth; actor/audit tracking. |
| Runs on your infrastructure | Any S3-compatible object store: on-prem via RustFS / MinIO, or AWS S3 / R2 / GCS. VPC, on-prem, hybrid; your data never leaves your store. |
| Open, versioned storage | Lance columnar format: branchable, time-travelable, with native blob-as-data (docs, images, video). |
What you can build
| Use case | What it's for |
|---|---|
| Company brain | Org knowledge unified into one graph every agent can query |
| Agentic memory | Durable, versioned memory: a branch per agent or per task, merged on review |
| Context graph | Decision traces and codified tribal knowledge for retrieval |
| Dev graph | Issues & dependency model that coding agents read and write |
| R&D / ML data layer | Experiments and trials written into branches, versioned for training & eval |
Install
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | bash
This installs omnigraph (CLI) and omnigraph-server into ~/.local/bin from
published release binaries. Or with Homebrew:
brew tap ModernRelay/tap
brew install ModernRelay/tap/omnigraph
Set it up with an AI agent
Omnigraph is built to be run by coding agents. Two ways in:
Teach your agent the playbook. This repo ships the
omnigraph agent skill: the operational playbook
covering cluster mode, the two config surfaces, schema evolution, query linting,
data writes, branches, Cedar policy, and the common gotchas.
npx skills add ModernRelay/omnigraph@omnigraph
Or have an agent set it up from scratch. Paste this into Claude Code, Codex, or any agent that can read a URL and run a shell command:
Help me set up Omnigraph
1. Read the docs at https://github.com/ModernRelay/omnigraph, starting with
docs/user/clusters/index.md, then docs/user/deployment.md.
2. Skim the starter graphs and seed data in the cookbooks:
https://github.com/ModernRelay/omnigraph-cookbooks
3. Ask me what I want to build (company brain, agent memory, dev graph,
research / R&D layer, …). Then stand up a cluster for it, load a little
data, and run a query so I can see it working.
For ready-to-run graphs with real seed data (company brain, VC operating system,
pharma & industry intel),
ModernRelay/omnigraph-cookbooks
is the fastest way to see Omnigraph shaped to a real domain.
Deploy
A deployment is a cluster: a multigraph config directory that declares
its graphs, schemas, stored queries, and policies as code. You manage it
Terraform-style: cluster plan previews the diff, cluster apply converges
it. omnigraph-server then boots from the cluster and brings every graph online
at /graphs/{id}/…, each behind its own policy.
1. Declare the cluster.
company-brain/
├── cluster.yaml
├── people.pg # schema for the "knowledge" graph
├── queries/ # stored queries: the .gq files ARE the declaration
│ └── people.gq
└── base.policy.yaml # a Cedar policy bundle
# cluster.yaml
version: 1
metadata:
name: company-brain
storage: s3://company/clusters/company-brain # ledger, catalog, and graph data live here
graphs:
knowledge:
schema: people.pg
queries: queries/ # every `query <name>` in queries/*.gq registers
policies:
base:
file: base.policy.yaml
applies_to: [knowledge] # graph-bound; use [cluster] for server-level
2. Stand up your object store. On-prem, run RustFS (or MinIO); Omnigraph
writes Lance to it over the standard S3
API. In the cloud, point the same AWS_* env at S3 / R2 / GCS instead.
3. Converge and run. apply creates each graph, applies its schema, and
publishes queries and policies into the content-addressed catalog. It is
idempotent; re-running is always safe.
omnigraph cluster validate # parse + typecheck everything
omnigraph cluster plan # preview what apply would do
omnigraph cluster apply # converge
# Boot the server from the cluster dir; storage resolves through cluster.yaml
omnigraph-server --cluster company-brain --bind 0.0.0.0:8080
See the cluster guide for the day-2 loop
(edit → plan → apply → restart), approval gates for destructive changes, drift
inspection, and recovery; the deployment guide for
containers, AWS/Railway, auth, and the full AWS_* contract.
Query and mutate
Set a default server and graph once in ~/.omnigraph/config.yaml, and the
everyday commands stay short. Stored queries and mutations run by name:
omnigraph query search_docs --params '{"q":"AI safety"}'
omnigraph mutate add_person --params '{"name":"Mina"}'
# Branch, review, merge across the whole graph; agents write in isolation
omnigraph branch create --from main agent/ingest-42
omnigraph branch merge agent/ingest-42 --into main
An alias is shorter still: bind a server, graph, and stored query to one
name, then omnigraph alias triage runs it. For an ad-hoc target, any command
still takes --server <name|url> --graph <id> (or --store <uri> for a local
graph). See the CLI reference.
Security & governance
- Engine-wide enforcement: every write path goes through the same Cedar gate, so the HTTP server, the CLI, and the embedded SDK obey identical rules.
- Declared in the cluster: a policy bundle is bound to graphs (or the whole server) via
policies:→applies_to. - Scoped: rules apply per graph, per branch, or server-wide.
- No plaintext tokens: bearer tokens are hashed at startup and compared in constant time.
- Forge-proof identity: the actor is resolved server-side from the token; clients can't set it.
See the policy guide.
Clients & SDKs
| Client | Use it for | Where |
|---|---|---|
| TypeScript SDK | typed access from Node / TS | @modernrelay/omnigraph · source |
| MCP server | bridge Omnigraph to LLM hosts (Claude, Codex, …) | @modernrelay/omnigraph-mcp |
| HTTP / OpenAPI | any language, the wire contract | the server's OpenAPI spec |
| Python SDK | typed access from Python | coming soon |
Both npm packages are versioned in lockstep with omnigraph-server.
Local quick test (no server)
1-min setup to try it: an embedded, local file-backed graph (no server, no object store). For dev and experiments; production is the deployed cluster above.
cat > schema.pg <<'PG'
node Signal { slug: String @key, title: String }
node Pattern { slug: String @key, name: String }
edge Indicates: Signal -> Pattern
PG
printf '%s\n' \
'{"type":"Signal","data":{"slug":"s1","title":"OSS model adoption surging"}}' \
'{"type":"Pattern","data":{"slug":"p1","name":"adoption"}}' \
'{"edge":"Indicates","from":"s1","to":"p1"}' > data.jsonl
omnigraph init --schema schema.pg ./graph.omni
omnigraph load --data data.jsonl --mode overwrite --store ./graph.omni
# "What pattern does signal s1 indicate?"
omnigraph query --store ./graph.omni \
-e 'query indicates() { match { $s: Signal { slug: "s1" } $s indicates $p } return { $p.name } }'
# → adoption
Docs
Build And Test
cargo build --workspace
cargo test --workspace
Notes:
- Rust stable toolchain, edition 2024
- CI runs
cargo test --workspace --locked - Full CI and some local test flows require
protobuf-compiler - S3 integration tests expect an S3-compatible endpoint such as RustFS
Workspace Crates
crates/omnigraph-compiler: shared schema/query parser, typechecker, catalog, and IR lowering (zero Lance dependency)crates/omnigraph(packageomnigraph-engine): storage/runtime, branching, merge, change detection, query execution, and embeddingscrates/omnigraph-policy: Cedar policy compilation and enforcementcrates/omnigraph-api-types: shared HTTP wire DTOs used by both the server and the CLIcrates/omnigraph-cluster: cluster config validation, planning, and apply (the control plane)crates/omnigraph-server: Axum HTTP server, cluster-first, runs N graphs under/graphs/{id}/…crates/omnigraph-cli: CLI for graph lifecycle, query/mutate, branch/commit/merge, schema/lint, snapshot/export, cluster control, policy/queries, profiles, and maintenance
Contributing
Please open an issue, spec, or design discussion before sending large code changes. Design feedback and concrete problem statements are the fastest way to collaborate on the roadmap.
Community
Join the Omnigraph Slack community to ask questions, share feedback, and follow development.