mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-06-12 01:45:14 +02:00
Lakehouse-native graph engine with git-style workflows
https://omnigraph.dev
3f21f59210
Wire the dormant DropProperty variant end-to-end for the Soft case. Per docs/schema-lint-v1-plan.md, commit #3 of the schema-lint chassis v1 series (MR-694). Planner (schema_plan.rs): - plan_properties: emit DropProperty { type_kind, type_name, property_name, mode: Soft } instead of UnsupportedChange when a property exists in accepted but not in desired. Plan is now supported = true for drop-only changes. Apply (schema_apply.rs): - Route DropProperty { Soft } through rewritten_tables. The existing batch_for_schema_apply_rewrite path already iterates the *target* schema fields, so a property absent from desired_catalog is naturally projected away. The prior Lance version retains the dropped column for time-travel reversibility (until cleanup runs). - DropType still errors (lands in commit #4 with different mechanics: __manifest entry removal instead of column projection). - DropProperty { Hard } still errors (lands in commit #5 with --allow-data-loss CLI flag + immediate compact_files + cleanup_old_versions). Tests: - Planner unit test plan_emits_soft_drop_for_removed_nullable_property asserts the variant emission + supported = true + no UnsupportedChange. - Integration test apply_schema_drops_a_nullable_property_softly_ preserves_prior_version (replaces the former apply_schema_rejects_dropping_a_property_with_data) asserts: (a) plan contains DropProperty { Soft } (b) apply succeeds + manifest advances + row count unchanged (c) current dataset schema lacks the dropped column (d) snapshot_at_version(pre_drop) still has the dropped column (e) reopen consistency — drop preserved across engine restart Recovery: rides on SidecarKind::SchemaApply per MR-847. No new sidecar kind needed; the entire apply path is already sidecar-wrapped. Substrate alignment: this commit uses the stage_overwrite full-rewrite path (full_rewrite cost class) rather than Lance native drop_columns (catalog_only cost class). MR-948 is the follow-up substrate-alignment refactor that introduces a LanceColumnOp surface and switches the metadata-only case onto drop_columns. Functional outcome is identical; cost-class improvement deferred. Test results: - cargo test -p omnigraph-compiler --lib: 238 passed - cargo test -p omnigraph-engine --test schema_apply: 11 passed 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|---|---|---|
| .cargo | ||
| .context | ||
| .github | ||
| crates | ||
| docker | ||
| docs | ||
| scripts | ||
| .dockerignore | ||
| .gitignore | ||
| AGENTS.md | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CLAUDE.md | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| Dockerfile | ||
| LICENSE | ||
| og-cheet-sheet.md | ||
| omnigraph.example.yaml | ||
| openapi.json | ||
| README.md | ||
| rust-toolchain.toml | ||
| SECURITY.md | ||
Omnigraph
Object-storage native graph engine with git-style workflows. Designed for agents as first-class operators.
Branch, commit, and merge typed graph data like source code. Multi-modal, self-hosted, open source.
Built on Rust, Arrow, DataFusion and Lance.
Join the Omnigraph Slack community
Use Cases
- Company brains / Second brains
- Context graphs
- Backbone for multi-agent research
- Incident response graphs
- Compliance & audit graphs
- Enterprise knowledge systems
Capabilities
- Typed schema, typed queries, and typed mutations
- Native blob-as-data support (docs, images, videos, etc)
- Schema-as-code, query validation and linting
- Git-style graph workflows: branches, commits, merges, and transactional runs
- Local, on-prem & cloud S3-native storage with snapshot-pinned reads
- Graph traversal + text, fuzzy, BM25, vector, and RRF search in one runtime
- Policy-as-code for server-side access control
- Single CLI for multiple deployments
Quick Install
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | bash
This installs omnigraph and omnigraph-server into ~/.local/bin from
published release binaries.
Or install with Homebrew:
brew tap ModernRelay/tap
brew install ModernRelay/tap/omnigraph
For starter graphs and agent skills to bootstrap and operate Omnigraph, see ModernRelay/omnigraph-cookbooks.
One-Command Local RustFS Bootstrap
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/local-rustfs-bootstrap.sh | bash
That bootstrap:
- starts RustFS on
127.0.0.1:9000 - creates a bucket and S3-backed repo
- loads the checked-in context fixture
- launches
omnigraph-serveron127.0.0.1:8080
Docker must be installed and running first.
The RustFS bootstrap prefers the rolling edge binaries and only falls back to
source builds when release assets are unavailable.
If a previous run left objects under the same repo prefix but did not finish
initializing the repo, rerun with RESET_REPO=1 or set PREFIX to a new
value.
Common Commands
The same URI works for local paths, s3://…, or http://host:port.
omnigraph init --schema ./schema.pg ./repo.omni
omnigraph load --data ./data.jsonl ./repo.omni
omnigraph read --query ./queries.gq --name get_person --params '{"name":"Alice"}' ./repo.omni
omnigraph change --query ./queries.gq --name insert_person --params '{"name":"Mina"}' ./repo.omni
omnigraph branch create --from main feature-x ./repo.omni
omnigraph branch merge feature-x --into main ./repo.omni
See docs/user/cli.md for schema apply, snapshots, ingest, runs, and policy commands.
Docs
Build And Test
cargo build --workspace
cargo check --workspace
cargo test --workspace
Notes:
- Rust stable toolchain, edition 2024
- CI runs
cargo test --workspace --locked - Full CI and some local test flows require
protobuf-compiler - S3 integration tests expect an S3-compatible endpoint such as RustFS
Workspace Crates
crates/omnigraph-compiler: shared schema/query parser, typechecker, catalog, and IR loweringcrates/omnigraph: storage/runtime, branching, merge, change detection, and query executioncrates/omnigraph-cli: CLI for init/load/ingest/read/change/branch/snapshot/export/policy operationscrates/omnigraph-server: Axum HTTP server for remote reads, changes, ingest, export, branches, commits, and runs
Contributing
Please open an issue, spec, or design discussion before sending large code changes. Design feedback and concrete problem statements are the fastest way to collaborate on the roadmap.
Community
Join the Omnigraph Slack community to ask questions, share feedback, and follow development.