mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-06-18 02:24:27 +02:00
ee4986e9a1
* docs: optimize README for dev onboarding; fix 0.7.0 staleness The README's setup half drifted from the shipped 0.7.0 CLI and led with the heaviest path (Docker + RustFS). This reworks it for fast, correct onboarding: README.md - New zero-dependency "Your first graph in 60 seconds" hero: a fully copy-pasteable local file-backed loop (schema → init → load → query → branch). - Add a correct "Serve it" section (cluster apply + omnigraph-server --cluster); the server is cluster-only on main, so the old positional-URI boot is gone. - Demote the RustFS bootstrap to "rehearse the S3 path locally"; reframe the storage bullet as "filesystem or any S3-compatible store (AWS S3, R2, MinIO, RustFS)" — RustFS is a provider, not a storage class. - Fix crate/MCP descriptions (query/mutate/load, not read/change/ingest). docs/user/quickstart.md - Fix the query example: `read --name <q> … <uri>` is removed — the query name is positional and the graph is addressed with `--store` (`omnigraph query find_people --query queries.gq --store graph.omni`). scripts/local-rustfs-bootstrap.sh - Convert to cluster mode: write a cluster.yaml (storage: s3://…), then validate → import → apply, load the fixture into the derived root with the now-required --mode, and serve with `omnigraph-server --cluster`. The old flow (`load` without --mode, `omnigraph-server <URI>` positional boot) no longer works on a cluster-only server. * docs: move agent skill into the repo, add agent-setup snippet, drop rustfs script skills/omnigraph - The operational skill (formerly `omnigraph-best-practices` in the cookbooks repo) now lives with the engine it documents, co-versioned. Renamed to `omnigraph`; repository metadata repointed here. - Broadened the description to trigger on intent — storing/retrieving/querying knowledge, agent memory, building a knowledge graph, operating Omnigraph — as well as on CLI/artifact sightings (stays ≤1024 chars). - Install: `npx skills add ModernRelay/omnigraph@omnigraph`. README - New "Set it up with an AI agent" paste snippet: installs the skill, reads the docs (URL), browses the cookbooks, and asks the user about a use case before standing up a first graph. - "Agent skill & starter graphs" section points at skills/omnigraph + cookbooks. Drop scripts/local-rustfs-bootstrap.sh - Not CI-tested (so it rotted: it broke on the cluster-only migration — positional server boot, load without --mode), demoed the now-optional S3 path, and was the most fragile artifact in the repo. Replaced with a "Testing against S3 locally" guide in deployment.md (docker run RustFS/MinIO + AWS_* env + cluster-on-S3). README/AGENTS references updated.
199 lines
9.1 KiB
Markdown
199 lines
9.1 KiB
Markdown
# Omnigraph
|
|
|
|
[](LICENSE)
|
|
[](rust-toolchain.toml)
|
|
[](https://crates.io/crates/omnigraph-cli)
|
|
[](https://github.com/ModernRelay/omnigraph/actions/workflows/ci.yml)
|
|
|
|
**Lakehouse native graph engine built for context assembly**
|
|
|
|
Omnigraph acts as operational state & coordination layer for agents.
|
|
Hundreds of agents can enrich the graph on parallel isolated branches and changes can be reviewed and merged safely.
|
|
|
|
- Git-style versioning & branching
|
|
- Multimodal retrieval (graph+vector/fts+filters) optimized for context assembly
|
|
- Runs on the local filesystem or any S3-compatible object store (AWS S3, R2, MinIO, RustFS)
|
|
- Native blob-as-data support (docs, images, videos, etc)
|
|
- VPC, On-prem, hybrid deployment
|
|
- [`Lance`](https://github.com/lance-format/lance) format as open storage layer
|
|
|
|
| AS CODE | What it means |
|
|
|---|---|
|
|
| **Schema AS CODE** | Typed `.pg` schemas, planned, applied, enforced |
|
|
| **Context AS CODE** | Linted queries & agentic nudges, versioned and reusable |
|
|
| **Security AS CODE** | Cedar policies enforced server-side on every mutation |
|
|
| **Dashboards AS CODE** | Declarative views & controls over the graph *(coming)* |
|
|
|
|
## Core Use Cases
|
|
|
|
| Use case | What it's for
|
|
|---|---|
|
|
| **Company brain** | Org knowledge unified into one queryable graph |
|
|
| **Context graph** | Decision traces and codified tribal knowledge |
|
|
| **Agentic memory** | Durable, versioned memory for long-running agents |
|
|
| **Dev graph** | Issues & dependency model for coding agents |
|
|
| **R&D data layer** | Experiments & trials data written into branches |
|
|
| **ML workflows** | Versioned, branchable graphs for training & eval |
|
|
| **Karpathy's LLM wiki** | A living, agent-updatable knowledge base |
|
|
|
|
## Quick Install
|
|
|
|
```bash
|
|
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | bash
|
|
```
|
|
|
|
This installs `omnigraph` and `omnigraph-server` into `~/.local/bin` from
|
|
published release binaries.
|
|
|
|
Or install with Homebrew:
|
|
|
|
```bash
|
|
brew tap ModernRelay/tap
|
|
brew install ModernRelay/tap/omnigraph
|
|
```
|
|
|
|
## Set it up with an AI agent
|
|
|
|
Omnigraph is built to be set up by coding agents. Paste this into Claude Code,
|
|
Cursor, or any agent that can read a URL, install a package, and run a shell
|
|
command — it installs the skill, reads the docs, and walks you through setup for
|
|
your use case:
|
|
|
|
```text
|
|
Help me set up Omnigraph (a lakehouse-native graph engine for agents).
|
|
|
|
1. Install the Omnigraph skill so you operate it correctly:
|
|
npx skills add ModernRelay/omnigraph@omnigraph
|
|
2. Read the docs at https://github.com/ModernRelay/omnigraph — start with
|
|
docs/user/quickstart.md, then docs/user/clusters/index.md.
|
|
3. Skim the starter graphs and seed data in the cookbooks:
|
|
https://github.com/ModernRelay/omnigraph-cookbooks
|
|
4. Ask me what I want to build (company brain, agent memory, dev graph,
|
|
research / R&D layer, …). Then install the CLI, stand up a first graph for
|
|
that use case, load a little data, and run a query so I can see it working.
|
|
```
|
|
|
|
Works with any agent that can browse a URL, install a package, and run a shell.
|
|
|
|
## Agent skill & starter graphs
|
|
|
|
This repo ships the [**`omnigraph` agent skill**](skills/omnigraph) — the
|
|
operational playbook (cluster mode, the two config surfaces, schema evolution,
|
|
query linting, data writes, branches, Cedar policy, and common gotchas) that
|
|
teaches a coding agent to drive Omnigraph correctly. Install it with:
|
|
|
|
```bash
|
|
npx skills add ModernRelay/omnigraph@omnigraph
|
|
```
|
|
|
|
For ready-to-run graphs with real seed data (company brain, VC operating system,
|
|
pharma & industry intel),
|
|
[`ModernRelay/omnigraph-cookbooks`](https://github.com/ModernRelay/omnigraph-cookbooks)
|
|
is the fastest way to see Omnigraph shaped to a real domain. To rehearse the S3
|
|
path locally, see [deployment.md → Testing against S3 locally](docs/user/deployment.md#testing-against-s3-locally).
|
|
|
|
## Common Commands
|
|
|
|
A deployment is a **cluster**. A `cluster.yaml` declares its graphs, schemas,
|
|
stored queries, and policies; you converge it with `cluster apply` and serve it.
|
|
The server is cluster-first — it boots only from a cluster and serves every graph
|
|
under `/graphs/{id}/…`. Day-to-day work goes through that server: graphs are
|
|
addressed with `--server <name|url>` (+ `--graph <id>`), and `query`/`mutate`
|
|
invoke a stored query from the catalog **by name**.
|
|
|
|
```bash
|
|
# 1. Converge the declared cluster, then serve it (--as attributes the apply)
|
|
omnigraph cluster apply --config ./company-brain --as you
|
|
omnigraph-server --cluster ./company-brain --bind 0.0.0.0:8080
|
|
# or config-free from object storage — the bucket IS the deployment:
|
|
# omnigraph-server --cluster s3://my-bucket/company-brain --bind 0.0.0.0:8080
|
|
|
|
# 2. Work against the served graph — stored queries invoked by name
|
|
omnigraph query find_people --server prod --graph knowledge --params '{"q":"AI safety"}'
|
|
omnigraph mutate add_person --server prod --graph knowledge --params '{"name":"Mina"}'
|
|
omnigraph load --data ./data.jsonl --mode merge --server prod --graph knowledge
|
|
|
|
# 3. Branch and merge, Git-style across the whole graph
|
|
omnigraph branch create --from main review/2026-06 --server prod --graph knowledge
|
|
omnigraph branch merge review/2026-06 --into main --server prod --graph knowledge
|
|
```
|
|
|
|
Set a default scope (or a `--profile`) in `~/.omnigraph/config.yaml` — operator
|
|
identity, named servers/clusters, credentials — and the `--server`/`--graph`
|
|
flags drop away (`omnigraph query find_people --params …`).
|
|
|
|
**Local / ad-hoc.** For quick iteration on a standalone graph (no cluster, no
|
|
server), address storage directly with `--store` (or a positional `file://` /
|
|
`s3://` URI) and run ad-hoc `.gq` with `--query` (the positional then selects
|
|
which query in the file):
|
|
|
|
```bash
|
|
omnigraph init --schema ./schema.pg ./graph.omni
|
|
omnigraph load --data ./data.jsonl --mode merge --store ./graph.omni
|
|
omnigraph query --query ./queries.gq get_person --params '{"name":"Alice"}' --store ./graph.omni
|
|
```
|
|
|
|
See [docs/user/cli/index.md](docs/user/cli/index.md), the
|
|
[CLI reference](docs/user/cli/reference.md), the
|
|
[cluster guide](docs/user/clusters/index.md), and the
|
|
[deployment guide](docs/user/deployment.md) for schema apply, snapshots, commits,
|
|
profiles, and policy/queries tooling.
|
|
|
|
## Clients
|
|
|
|
For programmatic access to a running `omnigraph-server`:
|
|
|
|
- **TypeScript SDK** — [`@modernrelay/omnigraph`](https://www.npmjs.com/package/@modernrelay/omnigraph) ([source](https://github.com/ModernRelay/omnigraph-ts/tree/main/packages/sdk)). Instance-per-client, typed errors, camelCase types, async-iterator streaming export.
|
|
|
|
```bash
|
|
npm install @modernrelay/omnigraph
|
|
```
|
|
|
|
- **Model Context Protocol server** — [`@modernrelay/omnigraph-mcp`](https://www.npmjs.com/package/@modernrelay/omnigraph-mcp) ([source](https://github.com/ModernRelay/omnigraph-ts/tree/main/packages/mcp)). Bridges Omnigraph to LLM hosts (Claude Desktop, Claude Code, …) over stdio. Exposes tools and resources for schema, branches, queries, mutations, ingest, and bundles curated best-practices guidance from the cookbook.
|
|
|
|
```bash
|
|
npm install -g @modernrelay/omnigraph-mcp
|
|
```
|
|
|
|
Both packages are versioned in lockstep with `omnigraph-server` on major.minor: `@modernrelay/omnigraph@X.Y.*` targets `omnigraph-server@X.Y.*`. See [`ModernRelay/omnigraph-ts`](https://github.com/ModernRelay/omnigraph-ts) for the monorepo.
|
|
|
|
## Docs
|
|
|
|
- [Install guide](docs/user/install.md)
|
|
- [Deployment guide](docs/user/deployment.md)
|
|
|
|
## Build And Test
|
|
|
|
```bash
|
|
cargo build --workspace
|
|
cargo check --workspace
|
|
cargo test --workspace
|
|
```
|
|
|
|
Notes:
|
|
|
|
- Rust stable toolchain, edition 2024
|
|
- CI runs `cargo test --workspace --locked`
|
|
- Full CI and some local test flows require `protobuf-compiler`
|
|
- S3 integration tests expect an S3-compatible endpoint such as RustFS
|
|
|
|
## Workspace Crates
|
|
|
|
- `crates/omnigraph-compiler`: shared schema/query parser, typechecker, catalog, and IR lowering (zero Lance dependency)
|
|
- `crates/omnigraph` (package `omnigraph-engine`): storage/runtime, branching, merge, change detection, query execution, and embeddings
|
|
- `crates/omnigraph-policy`: Cedar policy compilation and enforcement
|
|
- `crates/omnigraph-api-types`: shared HTTP wire DTOs used by both the server and the CLI
|
|
- `crates/omnigraph-cluster`: cluster config validation, planning, and apply (the control plane)
|
|
- `crates/omnigraph-server`: Axum HTTP server — cluster-first, serving N graphs under `/graphs/{id}/…`
|
|
- `crates/omnigraph-cli`: CLI for graph lifecycle (init/load), query/mutate, branch/commit/merge, schema/lint, snapshot/export, cluster control, policy/queries, profiles, and maintenance (optimize/repair/cleanup)
|
|
|
|
## Contributing
|
|
|
|
Please open an issue, spec, or design discussion before sending large code
|
|
changes. Design feedback and concrete problem statements are the fastest way to
|
|
collaborate on the roadmap.
|
|
|
|
## Community
|
|
|
|
Join the [Omnigraph Slack community](https://join.slack.com/t/omnigraphworkspace/shared_invite/zt-3wfpglyxj-lHvJGhuySPfqLtN35uJZNw)
|
|
to ask questions, share feedback, and follow development.
|