Lakehouse-native graph engine with git-style workflows https://omnigraph.dev
Find a file
Andrew Altshuler 58c66a54a2
docs(cluster): RFC-004 — graph & schema apply design (Phase 4) (#168)
* docs(cluster): RFC-004 — graph & schema apply design (Phase 4)

The design the implementation spec's exit criteria require before
graph-moving cluster apply ships. Core positions:

- Cluster recovery is roll-forward-only: the engine's own sidecars make every
  graph-level operation atomic within the graph, so the cluster never rolls a
  graph back — its sidecars (__cluster/recoveries/{ulid}.json) classify and
  record, converging the ledger to observable reality (axiom 5) or surfacing
  a loud pending-repair condition. Eight-row decision matrix, every row
  testable with the Stage 3B failpoint harness.
- Irreversible operations (graph delete, allow_data_loss schema apply)
  consume digest-bound approval artifacts written by a new cluster approve
  command and retired into state.approval_records (axiom 11). A stale
  approval can never authorize a different change.
- cluster apply gains an actor, threaded to apply_schema_as so engine Cedar
  enforcement and commit attribution work unchanged; the cluster adds no
  policy engine of its own.
- Deterministic ordering (creates -> schema applies -> catalog -> deletes),
  per-resource apply groups, cross-graph atomicity explicitly not promised.
- Staged 4A graph create / 4B schema apply / 4C graph delete, each gated on
  per-matrix-row failpoint tests.

Answers exit criteria 2 and 4 fully, 1/5/6 partially; 3/7/8/9 deferred to
their phases (coverage table in the RFC). Linked from the dev index and the
implementation spec's Phase 4 section.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(cluster): RFC-004 review fixes — graph_delete sweep rows, state_cas_base contract

Two greptile findings: (1) D3 row 2 could not be evaluated for graph_delete
(no manifest to version-check after prefix removal) and 'root absent, state
already tombstoned' fell into the stale row — split into rows 7 (delete's
analog of row 2) and 7b (the roll-forward), with expected_manifest_version
documented as always null for the delete kind. (2) state_cas_base is now
explicitly audit/diagnostics-only — the sweep never consults it; independent
state mutations are handled by the ordinary CAS like any concurrent write.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 04:34:14 +03:00
.cargo Raise LANCE_MEM_POOL_SIZE to 1 GB in .cargo/config.toml 2026-04-19 22:27:49 +03:00
.context Investigate Lance MergeInsertBuilder CAS granularity (MR-766 prereq) 2026-04-28 23:30:17 +00:00
.github test(cluster): failpoint tests for crash-mid-apply and state CAS race 2026-06-10 02:14:06 +03:00
crates fix(cluster): RAII-guard the callback failpoint 2026-06-10 02:36:24 +03:00
docker feat(server): compose OMNIGRAPH_TARGET_URI with OMNIGRAPH_CONFIG in entrypoint (#129) 2026-05-30 20:17:55 +01:00
docs docs(cluster): RFC-004 — graph & schema apply design (Phase 4) (#168) 2026-06-10 04:34:14 +03:00
scripts governance: external contribution model (issues/discussions/RFCs/PRs) (#143) 2026-06-06 23:58:08 +03:00
.dockerignore Initial public Omnigraph repository 2026-04-10 20:49:41 +03:00
.gitignore release: v0.5.0 (#115) 2026-05-23 13:59:42 +01:00
AGENTS.md (feat) convert engine call sites to &dyn TableStorage; demote legacy TableStore methods to pub(crate) (#86) 2026-06-09 23:03:08 +02:00
Cargo.lock build: commit Cargo.lock for omnigraph-cluster's optional fail dependency 2026-06-10 02:21:10 +03:00
Cargo.toml feat(cluster): add read-only validate and plan 2026-06-08 20:07:39 +03:00
CLAUDE.md Add AGENTS.md as canonical agent guide; symlink CLAUDE.md to it 2026-04-28 23:10:09 +02:00
CODE_OF_CONDUCT.md Initial public Omnigraph repository 2026-04-10 20:49:41 +03:00
CONTRIBUTING.md governance: external contribution model (issues/discussions/RFCs/PRs) (#143) 2026-06-06 23:58:08 +03:00
Dockerfile Dockerfile: switch base from Docker Hub to ECR Public 2026-04-20 13:46:23 +03:00
GOVERNANCE.md governance: external contribution model (issues/discussions/RFCs/PRs) (#143) 2026-06-06 23:58:08 +03:00
LICENSE Initial public Omnigraph repository 2026-04-10 20:49:41 +03:00
og-cheet-sheet.md feat: inline query strings in CLI and HTTP server (#110) 2026-05-29 13:41:54 +02:00
omnigraph.example.yaml example config: use graphs / cli.graph, matching the MR-603 rename 2026-04-18 23:40:35 +03:00
openapi.json release: v0.6.2 2026-06-09 15:59:59 +02:00
README.md README: link to TypeScript SDK and MCP server (#92) 2026-05-30 14:29:49 +02:00
rust-toolchain.toml Initial public Omnigraph repository 2026-04-10 20:49:41 +03:00
SECURITY.md Initial public Omnigraph repository 2026-04-10 20:49:41 +03:00

Omnigraph

License: MIT Rust Crates.io CI

Lakehouse native graph engine built for context assembly

Omnigraph acts as operational state & coordination layer for agents

  • Git-style versioning & branching
  • Multimodal retrieval (graph+vector/fts+filters) optimized for context assembly
  • Object storage native (S3, RustFS)
  • Native blob-as-data support (docs, images, videos, etc)
  • VPC, On-prem, hybrid deployment
  • Lance format as open storage layer
AS CODE What it means
Schema AS CODE Typed .pg schemas, planned, applied, enforced
Context AS CODE Linted queries & agentic nudges, versioned and reusable
Security AS CODE Cedar policies enforced server-side on every mutation
Dashboards AS CODE Declarative views & controls over the graph (coming)

Core Use Cases

Use case What it's for
Company brain Org knowledge unified into one queryable graph
Context graph Decision traces and codified tribal knowledge
Agentic memory Durable, versioned memory for long-running agents
Dev graph Issues & dependency model for coding agents
R&D data layer Experiments & trials data written into branches
ML workflows Versioned, branchable graphs for training & eval
Karpathy's LLM wiki A living, agent-updatable knowledge base

Quick Install

curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | bash

This installs omnigraph and omnigraph-server into ~/.local/bin from published release binaries.

Or install with Homebrew:

brew tap ModernRelay/tap
brew install ModernRelay/tap/omnigraph

For starter graphs and agent skills to bootstrap and operate Omnigraph, see ModernRelay/omnigraph-cookbooks.

One-Command Local RustFS Bootstrap

curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/local-rustfs-bootstrap.sh | bash

That bootstrap:

  • starts RustFS on 127.0.0.1:9000
  • creates a bucket and S3-backed graph
  • loads the checked-in context fixture
  • launches omnigraph-server on 127.0.0.1:8080

Docker must be installed and running first.

The RustFS bootstrap prefers the rolling edge binaries and only falls back to source builds when release assets are unavailable.

If a previous run left objects under the same graph prefix but did not finish initializing the graph, rerun with RESET_REPO=1 or set PREFIX to a new value.

Common Commands

The same URI works for local paths, s3://…, or http://host:port.

omnigraph init   --schema ./schema.pg ./graph.omni
omnigraph load   --data   ./data.jsonl ./graph.omni
omnigraph read   --query  ./queries.gq --name get_person --params '{"name":"Alice"}' ./graph.omni
omnigraph change --query  ./queries.gq --name insert_person --params '{"name":"Mina"}' ./graph.omni
omnigraph branch create --from main feature-x ./graph.omni
omnigraph branch merge  feature-x --into main ./graph.omni

See docs/user/cli.md for schema apply, snapshots, ingest, commits, and policy commands.

Clients

For programmatic access to a running omnigraph-server:

  • TypeScript SDK@modernrelay/omnigraph (source). Instance-per-client, typed errors, camelCase types, async-iterator streaming export.

    npm install @modernrelay/omnigraph
    
  • Model Context Protocol server@modernrelay/omnigraph-mcp (source). Bridges Omnigraph to LLM hosts (Claude Desktop, Claude Code, …) over stdio. Exposes tools and resources for schema, branches, queries, mutations, ingest, and bundles curated best-practices guidance from the cookbook.

    npm install -g @modernrelay/omnigraph-mcp
    

Both packages are versioned in lockstep with omnigraph-server on major.minor: @modernrelay/omnigraph@X.Y.* targets omnigraph-server@X.Y.*. See ModernRelay/omnigraph-ts for the monorepo.

Docs

Build And Test

cargo build --workspace
cargo check --workspace
cargo test --workspace

Notes:

  • Rust stable toolchain, edition 2024
  • CI runs cargo test --workspace --locked
  • Full CI and some local test flows require protobuf-compiler
  • S3 integration tests expect an S3-compatible endpoint such as RustFS

Workspace Crates

  • crates/omnigraph-compiler: shared schema/query parser, typechecker, catalog, and IR lowering
  • crates/omnigraph: storage/runtime, branching, merge, change detection, and query execution
  • crates/omnigraph-cli: CLI for graph lifecycle (init/load/ingest), query/mutate, branch/commit/merge, schema/lint, snapshot/export, policy, and maintenance (optimize/cleanup)
  • crates/omnigraph-server: Axum HTTP server for remote reads, changes, ingest, export, branches, and commits

Contributing

Please open an issue, spec, or design discussion before sending large code changes. Design feedback and concrete problem statements are the fastest way to collaborate on the roadmap.

Community

Join the Omnigraph Slack community to ask questions, share feedback, and follow development.